Professional Documents
Culture Documents
Abstract
This document describes the results of performance tests conducted for Ascentium Commerce Server 2009 R2 Enterprise Edition (Commerce Server, Commerce Server 2009 R2), using the SharePoint 2010 Solution Storefront site. This document describes the hardware and software configurations used for testing and provides a summary of the results.
Contents
Executive Summary ............................................................................................. 5
Performance Testing Overview .................................................................................................. 5 Topology ..................................................................................................................................... 5 Performance Testing .................................................................................................................. 5 Key Results ................................................................................................................................... 6 Hardware Details .......................................................................................................................... 7 Software Details ........................................................................................................................... 8
Configuration Details........................................................................................... 9
Data Configuration ....................................................................................................................... 9 Environment Configuration ......................................................................................................... 9 Test Rig ....................................................................................................................................... 9 Testing with SSL.................................................................................................................... 10 Database Servers ..................................................................................................................... 10 Presentation Server .................................................................................................................. 11 Application Server ..................................................................................................................... 11 Network ..................................................................................................................................... 12 Performance Monitoring Configuration ................................................................................... 12 Storage Configuration ............................................................................................................... 12 Storage Disk Layout.................................................................................................................. 13
Executive Summary
The Commerce Server product verification team conducted a series of performance tests using production grade hardware and the prescribed installations of the Commerce Server Solution Storefront site. All performance tests outlined in this document were performed on the same hardware lineup. This Executive Summary provides an overview of the tests performed, key test results, and a summary of software and hardware configurations. For the most current version of this document, see Technical Resources
Topology
Two-Tier Deployment: In a two-tier commerce deployment topology, the Commerce Server presentation and foundation software components are installed on the same Web server. This document refers to Web servers in a two-tier deployment as presentation Web servers. Two-tier commerce deployments include a presentation and data tier. Three-Tier Deployment: In a three-tier commerce deployment topology, the Commerce Server presentation and foundation components are installed on separate servers. This document refers to servers in a three-tier deployment by their respective role; Web servers that host the Commerce Server presentation software components are referred to as presentation servers. Application servers that host the Commerce Server foundation software components, allowing hosting of a Commerce Foundation service, are referred to as application servers. Three-tier commerce deployments include a presentation tier, an application tier, and a data tier.
Performance Testing
Performed testing against Commerce Server 2009 R2 Enterprise Edition deployed in a SharePoint Server 2010 farm environment in a two-tier configuration that consisted of: Four x64 presentation Web servers Two x64 database servers running SQL Server 2008 R2 Enterprise Edition One clustered modular storage array (with wide and automatic data striping)
Performed testing against Commerce Server 2009 R2 Enterprise Edition deployed in a farmbased SharePoint Server 2010 environment in a three-tier configuration that consisted of:
Four x64 Web servers in the presentation tier Two x64 application servers in the application tier Two x64 database servers running SQL Server 2008 R2 Enterprise Edition in the data tier One clustered modular storage array (with wide and automatic data striping)
Simulated 100 users per presentation server without incorporating think time emulation (simulated users clicked through their scenarios much faster than human users would do so). Testing was conducted over a one-hour period with a 5-minute warm-up. The simulated usage was divided into the following scenarios, according to the percentages shown in the following table:
Percentage Default site page(s) in this scenario
Scenario
Home Page Browse category Browse product Anonymous checkout Add to Wish List
10% 30 % 40 % 18 % 2%
Key Results
The following table summarizes the key results obtained by performing the same test run using the Solution Storefront site provided with Commerce Server 2009 R2. All tests were performed using 100 users per presentation server (without incorporating think time).
Test metric Two-tier Commerce Server 2009 R2 (Enterprise edition) deployed in a SharePoint Server 2010 farm
Approximate number of transactions (orders) per second # Web requests per seconds for the entire farm Average Response Time
5.38 orders/second
Test metric
Three-tier Commerce Server 2009 R2 (Enterprise edition) deployed in a SharePoint Server 2010 farm
Approximate number of transactions (orders) per second # Web requests per seconds for the entire farm Average Response Time
4.18 orders/second
109 requests/second
3.30 seconds
Hardware Details
The following table describes the computers that were used for the Web and database servers for the test run, and the SAN and networks used.
Server role Specification/configuration
HP ProLiant DL385 Two 2.4 GHz Opteron dual core processors 8 GB RAM Two 146 GB Ultra320 10K HDD
Database server
Dell PowerEdge R610 servers: Two 3.0 GHz Intel Xeon X5570 quad core processors with Hyper Threading 16 GB RAM Two Emulex LP9002 Database server connected to an EMC SAN with two connections F5 BIG-IP Local Traffic Manager One GBPS Ethernet on the Internet-facing side of the Web servers One GBPS Ethernet between the Web servers and the database server
Presentation server
Dell PowerEdge R610 servers: Two 3.0 GHz Intel Xeon X5570 quad core processors with Hyper Threading
Server role
Specification/configuration
SAN
EMC CLARiiON CX3-80: 16 GB Cache 8 4 GB/s Front-end Port 80 73 GB Fiber Channel 15K RPM Drives, 20 300 GB Fiber Channel 10K PRM Drives 2 Brocade Silkworm 4100s Two LUNs (configured RAID 1+0, one for logs, one for data)
Software Details
The following table provides an overview of the software installed on servers in each tier of a distributed commerce deployment (presentation, application and data tier.)
Computer role Software configuration
Microsoft Windows Server 2008 R2 Enterprise Edition with SP1 Microsoft Internet Services (IIS) 7.5 (Integrated Mode) Commerce Server 2009 R2 Enterprise Edition (Commerce Foundation component) Windows Server 2008 R2 Enterprise Edition with SP1 Microsoft SQL Server 2008 Enterprise R2 Microsoft Windows Server 2008 R2 Enterprise Edition with SP1 Microsoft Internet Information Services (IIS) 7.5 (Integrated Mode) Commerce Server 2009 R2 Enterprise Edition (three-tier and two-tier deployment) Note: In a two-tier commerce deployment, the presentation tier hosts Commerce Server 2009 R2 Presentation and
Database server
Presentation server
Computer role
Software configuration
Commerce Foundation component. In a three-tier commerce deployment, the presentation server hosts the presentation component only. Microsoft SharePoint Server Enterprise 2010 or SharePoint Foundation 2010
Configuration Details
The following environment, configurations, and settings were used to test and optimize the performance of Commerce Server 2009 R2 Solution Storefront site
Data Configuration
The data setup for the test environment consisted of the following: one catalog containing 1,000,000 products 100 million profile records broken down as follows: 10 million users, each configured with one shipping address and four credit cards with one billing address 40 million credit cards 50 million addresses
Environment Configuration
Test Rig
Commerce Server 2009 R2 software was installed and the Solution Storefront site deployed according to published documentation. The test environment was deployed in the lab on a private network. The load generation scripts were created using Microsoft Visual Studio Team System 2008 SP1 Web Tests. The test harness was comprised of one Visual Studio Team System 2008 SP1 load controller and four (4) Visual Studio Team System 2008 SP1 load agents. The following table details the configuration used for testing:
Client
Processors
Clock (GHz)
Architecture
AMD Opteron 880 Intel Core 2 Duo E6750 Intel Core 2 Duo E6750 Intel Core 2 Duo E6750 Intel Core 2 Duo E6750
Database Servers
The MSCS_CatalogScratch database data file and log file were each split into eight files of equal size on the SAN shared disk, one data file and one log file for each processor core (Note: hyperthreading cores should not be counted). Database administration activities were not performed during the test run. Similarly, for best performance, avoid such activities during peak hours. Catalog and inventory database statistics were always updated after large data import operations. Note For optimal performance, you should keep the statistics on the Commerce Server database tables up-to-date. You can do this by executing the Transact-SQL statement UPDATE STATISTICS WITH FULLSCAN on each large table in your Commerce Server databases. Full text searching was configured to use the SAN disk. EMC MetaLUN was used to stripe over a larger set of disks. Databases can be separated into multiple SQL Server instances to effectively reduce data load impact on a particular SQL Server instance. Testing that involved Commerce Server 2009 R2 Enterprise Edition demonstrated this configuration by splitting required databases over two SQL Server instances; this document refers to these databases as Catalog SQL Server and Transaction SQL Server. The following database layouts were used when two database servers were configured:
10
- Catalog SQL Server: Catalog, MSCS_Admin, Marketing, TransactionConfig, Inventory, CatalogScratch, and SharePoint 2010 databases. - Transaction SQL Server: Transactions, Profiles databases.
Presentation Server
IIS 7.5 application was configured to use integrated mode, as required for SharePoint Server 2010. The registry value UnsafeAuthenticatedConnectionSharing was changed from its default value of zero (0) to the value one (1). This change causes the Commerce Server 2009 R2 Profiles System to reuse database connections and prevents frequent log on and log off operations that would otherwise adversely affect performance. However, in some scenarios, there are security implications to consider. For more information about these security implications, see "Performance Issues Using SQL Server Windows Authentication" at Performance Issues Using SQL Server Windows Authentication. The registry location of this setting is: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\CommerceServer\9.0\Profi les,UnsafeAuthenticatedConnectionSharing = 1 Note You must run IISReset for this change to become effective.
Application Server
IIS 7.5 application was configured to use integrated mode. Three-tier testing utilized WSHttpBinding for all testing covered in this document, since it provided the ability for load balancing without sticky sessions, while maximizing security between the presentation and application tiers. WCF offers many other binding methods that in some cases can be utilized to improve performance. Consider load balancing, security, and performance requirements when choosing the appropriate type of WCF bindings for a deployment. For more information about WCF bindings, see WCF Bindings The registry value UnsafeAuthenticatedConnectionSharing was changed from its default value of zero (0) to the value one (1). This change causes the Commerce Server 2009 R2 Profiles System to reuse database connections and prevents frequent log on and log off operations that would otherwise adversely affect performance. However, in some scenarios, there are security implications to consider. For more information about these security implications, see "Performance Issues Using SQL Server Windows Authentication" at Performance Issues Using SQL Server Windows Authentication The registry location for this setting is: HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\CommerceServer\9.0\Profi les,UnsafeAuthenticatedConnectionSharing = 1
11
Note You must run IISReset for this change to become effective
Network
Use a hardware load balancer to distribute load to many presentation servers configured in a farm. Use a hardware load balancer to distribute load to many Commerce Foundation servers when using a three-tier commerce deployment topology. Use SSL offloading solution wherever possible. Deploy a firewall with at least 1Gbps throughput. Deploy switches with at least 1Gbps throughput. In three-tier commerce deployments, special attention is required to ensure that servers hosting an instance of the Commerce Foundation have sufficient network capabilities to handle expected load. Expected load depends on many factors such as the number of presentation Web servers in the farm, the use of rich Internet applications (RIA), and the number of applications sharing the server(s) hosting the Commerce Foundation instance(s). Increasing the network load capabilities of a Commerce Foundation instance may be achieved by increasing the bandwidth capabilities of the server network interface cards (NIC); adding additional network interface cards, allowing NIC load balancing; using different WCF bindings to improve data volume traveling over the network; and/or using multiple Commerce Foundation servers in a farm configuration.
Storage Configuration
Install two or more host bus adapters in a database server and use PowerPath/multipath I/O. Use mesh-networking topology for SAN fiber switches. Use a SAN that is optimized for write operations.
12
Processor assignments:
Processor B
Detailed Results
Two-Tier Commerce Server 2009 R2 (Enterprise Edition) with SharePoint Server 2010 Farm
Topology
Simulated usage was divided into the following scenarios, according to the percentages shown in the following table:
Percentage Default site page(s) in this scenario Transaction type
1
Scenario
10% 30 % 40 % 18 %
2%
Transactional
Test Results
The following test results should be considered "detailed" only with regard to the summaries that were presented earlier in this document. For ease of understanding, results represent averages over several different servers and have been rounded to a reasonable degree of precision. Test results are divided into four categories: summary, server use, database operations, and server-side (ASP.NET) metrics.
Result Summary
The following table summarizes the performance test results.
Test metrics Result
Approximate number of transactions (orders) per second Average response time Overall Average response time Non-transactional
5.38 orders/second
For data consistency and service reliability, some critical commerce operations are configured as transactional. In such
cases, the operation service treats the entire operation sequence as a single transaction. All the sequence components within the operation sequence that implement the IOperationSequenceComponent.Execute interface method must succeed entirely before any change is committed to the database.
14
Test metrics
Result
Average response time Transactional # Web Requests per second for the farm (per presentation Web server)
Note: excludes static content such as images, CSS
Server Use
The following table shows server use for the test run.
Server role CPU utilization Memory use
Catalog database disk Catalog database log disk Transactions database disk Transactions database log disk
0 0 12 0
85 614 308 72
35
Web Service\Get Requests/sec + Web Service\Post Requests/sec ASP.NET Apps v2.0.50727(aspnet_wp)\Request Time Execution
870
15
Average value
Counter name
0 48
Three-Tier Commerce Server 2009 R2 (Enterprise Edition) with SharePoint Server 2010 Farm
Topology
16
Scenario
Percentage
Transaction type
Home Page Browse category Browse product Anonymous checkout Add to Wish List
10% 30 % 40 % 18 %
2%
Transactional
Test Results
The following results should be considered "detailed" only with regard to the summaries that were presented earlier in this document. For ease of understanding, results represent averages over several different servers and have been rounded to a reasonable degree of precision. Test results are divided into four categories: summary, server use, database operations, and server-side (ASP.NET) metrics.
Result Summary
The following table summarizes the performance test results.
Test metrics Result
Approximate number of transactions (orders) per second Average response time Overall Average response time Non-transactional Average response time Transactional
4.18 orders/second
For data consistency and service reliability, some critical commerce operations are configured as transactional. In such
cases, the operation service treats the entire operation sequence as a single transaction. All the sequence components within the operation sequence that implement the IOperationSequenceComponent.Execute interface method must succeed entirely before any change is committed to the database.
17
Test metrics
Result
# Web Requests per second for the farm (per presentation Web server)
Note: excludes static content such as images, CSS
Server Use
The following table shows server use for the test run.
Server role CPU utilization Memory use
Presentation Web servers Application Web servers Catalog database server Transactions database server
Catalog database disk Catalog database log disk Transactions database disk Transactions database log disk
7.2 0 28 0
34 443 160 55
28.4
18
Average value
Counter name
1, 181
ASP.NET Apps v2.0.50727(aspnet_wp)\Request Time Execution ASP.NET v2.0.50727\Request Wait Time ASP.NET v2.0.50727\Requests Queued
0 65
Average value
Counter name
27
Web Service\Get Requests/sec + Web Service\Post Requests/sec ASP.NET Apps v2.0.50727(aspnet_wp)\Request Time Execution ASP.NET v2.0.50727\Request Wait Time ASP.NET v2.0.50727\Requests Queued
71
0 0
Conclusion
For each of the deployment configurations within this performance guide, user load pushed the presentation Web servers to a very high utilization level (over 90 percent processor time) to demonstrate a period of heavy demand. However, database server utilization remained low while load was applied, less than 10% of available CPU time, suggesting additional presentation Web servers could be included in the deployment to increase capacity without modification of the database or disk configuration. Similarly, in a three-tier deployment configuration, a limited number of additional presentation Web servers can be added with no changes required for the application server farm. Results also indicate three-tier deployments require a greater number of presentation Web servers than similar two-tier deployments to overcome the performance impact resulting from the use of a WCF service to handle Commerce Server requests.
19
20
The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos, people, places, and events depicted in examples herein are fictitious. No association with any real company, organization, product, domain name, e-mail address, logo, person, place, or event is intended or should be inferred. This document does not provide you with any legal rights to any intellectual property in any Microsoft product. You may copy and use this document for your internal, reference purposes. 2012 Microsoft. All rights reserved. Used with permission by Ascentium Corporation Microsoft, Active Directory, ActiveX, BackOffice, Bing Maps, BizTalk, Developer Studio, Expression Blend, Expression Web, FrontPage, Hotmail, Internet Explorer JScript, MSDN, MSDOS, MSN, Outlook, PivotChart, PivotTable, PowerShell. Silverlight, SharePoint, SQL Server, Visual Basic, Visual C++, Visual C#, Visual Studio, Webdings, Win32, Windows, Windows 7, Windows Live, Windows Media, Windows Server, and Windows Vista are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective owners.
21