• 1. WAS Performance Tuning (WAS 性能调优)
  • 2. AgendaOverview(概述) Tuning policies(调优策略) Tools(工具) PMI TPV Performance Advisor Request Metrics Case study DayTrader application Basic tuning Advanced tuning
  • 3. Key measurementsResponse time Time requested to process a request from a user Throughput How many requests/second, pages/second, or hits/second the site support Response time also a consideration Often, site runs below capacity to meet response time goals Users How many users the site can support concurrently User is a broad term, including Active user, making request to the site, i.e., click the button Thinking user, in think time Inactive user, left the site, but session data still present
  • 4. (本页无文本内容)
  • 5. Determine where’s bottleneckSystem is just as fast as its slowest component Eliminate bottlenecks by severity After addressing one bottleneck, retest to Measure improvement Find next bottleneck Some bottlenecks might be in system, some in middleware, and some in applications
  • 6. AgendaOverview Tuning policies Tools PMI TPV Performance Advisor Request Metrics Case study DayTrader application Basic tuning Advanced tuning
  • 7. Tuning JVM Heap Size(调整JVM堆大小)Always keep the JVM(s) within the physical memory Some starting points: -Xmx=512M for WAS Minimum/maximum heapsize Production systems set –Xms lower than –Xmx Gives headroom for emergencies In theory, results in more efficient object table “Burst” testing requires –Xms=-Xmx,Smaller heap sizeShorter but frequent GCsLarge heap sizeLess frequent but longer GCs
  • 8. Tuning GC(调整垃圾收集)Improved Garbage Collection features in WAS V6.1 Memory management is configurable using four different policies with varying characteristics 1. Optimize for Throughput – flat heap collector focused on maximum throughput 2. Optimize for Pause Time – flat heap collector with concurrent mark and sweep to minimize GC pause time 3. Generational Concurrent – divides heap into “nursery” and “tenured” segments providing fast collection for short lived objects. Can provide maximum throughput with minimal pause times 4. Subpool – a flat heap technique to help increase performance on multiprocessor systems , commonly greater than 8. Available on IBM pSeries™and zSeries™
  • 9. Web Container Tuning(调整WEB容量)Check if WebContainer thread pool set properly Sometimes, We need to increase the thread pool in order to keep the CPU entertained. But remember, a CPU can only drive 50 to 75 Java threads. Detecting if a thread is hung or just taking a long time to respond is a difficult problem to solve correctly HTTP transport persistent connections (KeepAlive) MaxKeepAliveRequests KeepAliveTimeout
  • 10. Database Connection Pooling(数据连接池)Use the Type 4 (pure Java) JDBC driver, if possible Type 4 JDBC drivers generally perform faster than Type 2 JDBC drivers. Type 4 drivers best suited for client/ server environment Connection pool tuning Min 1, Max 30 good starting point for tuning Maximum connections For optimal performance, set the value for the connection pool lower than the value for the Web container threadpool size. Minimum connections Until this number is exceeded, the pool maintenance thread does not discard physical connections. Connection properties tuning Statement cache size (per connection) Be aware, that specifying a larger statement cache size than needed wastes application memory and does not improve performance. Zero means there is no cache statement.
  • 11. Dynamic Caching(动态高速缓存)Better performance (less CPU), but higher memory consumption!
  • 12. Dynamic cache best practicesConsider Dynamic Caching during application design Part of design rather than retrofitting later Organize page layout and design for caching Servlet/JSP caching to hold repetitive, expensive page elements Cache those data which are expensive to reproduce and do not change frequently
  • 13. 64-bit WASBenefits: heap size > 2G (limit of 32-bit) Applications having large working set Applications having big cache scenario Applications having intensive and high precision computations Application heavy in statistics, security, encryption, simulations, etc) Is WAS 64-bit for me? Does your application need a Java heap much greater than ~2GB for higher performance? If Yes -> WAS 64-bit Does your application use computationally intensive algorithms which can benefit from high precision computation support? If Yes -> WAS 64-bit Does your application need a Java heap just a little more than ~2GB, i.e. 2~3GB? If Yes -> WAS 32-bit on 64-bit platforms
  • 14. Agendaoverview policies Performance tools PMI(性能监控基本结构) TPV Performance Advisor Request Metrics Case study DayTrader application Basic tuning Advanced tuning
  • 15. PMI (2) – PMI grouping and Performance ImpactsBasic: J2EE components, CPU usage, HTTPSession info (Default) Extended: Basic + additional Application Server resources (WLM, Performance advisor, Dynamic Cache, etc) All: Enables all statistics Custom: Fine-grained controlVaries depending on counter Basic has overhead of <2% Extended has overhead of <3% All has overhead of <6% Collection of JVMPI data adds additional overhead Collecting just GC and Thread data adds <2% Collecting all JVMPI data adds 20-25%
  • 16. PMI (3) - Performance Monitoring Applications
  • 17. TPV(性能查看器) (1)TPV: Monitoring data TPV: Logging data
  • 18. TPV (3) - Summary Report(摘要报告)Summary Reports - Provide quick and easy access to important metrics for key resource categories: Servlets: Lists key metrics for all servlets that are running in the application server, including: Total number of requests Average response time EJBs: Lists key metrics for all enterprise beans running in the server, including: Amount of time spent in their methods Number of EJB invocations Total time spent in each enterprise bean EJBMethods: Shows statistics for each EJB method Connection Pool: Lists all data source connections that are defined in the application server, and shows their usage over time Thread Pool: Shows the usage of all thread pools in the application server over time
  • 19. TPV (4) – Performance modules(性能模块)Available performance modules include: JDBC Connection Pools Usage information for JDBC connection pools JVM Runtime Memory usage and JVMTI data Servlet Session Manager Usage information for HTTP sessions Web Applications Response time and usage of individual servlets System Data CPU usage Thread Pools Transaction Manager In the TPV navigator, check the modules to monitor Corresponding PMI modules need to be enabled before they can be checked for monitoring
  • 20. TPV (5) – Performance modules example (性能模块 举例)
  • 21. Performance Advisor(顾问程序) Performance Advisor Provide configuration advice based on collected PMI data Advice based on basic rules of thumb for tuning WAS Rules are IBM defined and non-configurable Does not automatically tune appserver based on advice Administrator must manually apply recommendations All suggestions should be fully tested one-by-one to verify that performance is positively impacted. Suggested settings need to be checked against baseline performance to verify improvement
  • 22. TPV Performance Advisor output example (顾问程序 举例)
  • 23. Performance advisors best practicesShould only be used during stable production load tests Application needs to remain stable during production tests Any exceptions and deadlock issues need to be resolved prior to running The test load should be consistent Varied load might lead to contradictory advice Should be enabled once production load tests have reached peak load levels Exclude ramp-up and ramp-down times from monitoring Increasing or decreasing loads might lead to contradictory advice Certain types of advice only generated when CPU is being stressed Important: tune your application before you tune WebSphere
  • 24. Request Metrics (1) (请求度量)Request metrics track individual transactions Monitor transactions as they move through different components in WAS environment Monitor not only the time spent running the whole request, but also the time spent running request in each of the supported WAS components. Components supported: Web Server plug-ins, Web container, EJB container, JDBC calls, Web Service engine, JMS messaging engine, AsyncBeans Not limited to monitor transactions that occur on just one appserver, can also monitor transactions as they fall out across websphere appserver environment. RM data differs from PMI data. RM data is Transaction aware, while PMI data is not. PMI collects performance data for all current running transactions, while RM collects performance data for individual transactions.
  • 25. Request Metrics (2)Request Metrics Output To Logs http requests output to http_plugin.log Servlet/Web Services/EJB/JDBC/JMS requests output to SystemOut.log To ARM agent Then, visualized using TMTP, EWLM, other 3rd ARM management software To either location or both at the same time Recommended practice – disable request metrics logging when implementing an ARM agent to reduce disk I/O
  • 26. Request Metrics (3)Filters – limit the amount of transactions that are traced Trace levelShow Request Metrics
  • 27. Agendaoverview policies Performance tools PMI TPV Performance Advisor Request Metrics Case study(案件 学习) DayTrader application Basic tuning Advanced tuning
  • 28. DayTrader Overview Stock trading application – users login/logout, view portfolio, look up stock quotes, buy and sell stock share, and manage account information. Built on J2EE – JSP, Servlet, EJB, JMS, JDBC
  • 29. Basic tuning(基本调优)JVM heap size(JVM堆大小) Thread pool size(线程池大小) Connection pool size(连接池大小) Data source statement cache size (数据源声明缓存大小) ORB pass by reference(ORB 引用传递)
  • 30. JVM heap size(JVM堆大小)Default: -Xms50m –Xmx250m Servers => Application servers => server_name => Process definition => Java Virtual Machine Testing 256M 512M 1024M 2048M
  • 31. Check memory leak(检查 内存是否泄漏)If “free heap after collection ” reaches a steady state?
  • 32. GC frequency(垃圾收集 频率)
  • 33. GC pause time(垃圾收集 停顿时间)
  • 34. Keep Balance(保持平衡)%Time in GC VS Throughput1024M is a good recommendation
  • 35. Performance benefit of tuned JVM heap size
  • 36. Thread pool size(线程池大小)Three pools Web container – HTTP requests Default - MDB request ORB – EJB request Goal – eliminating the need to wait for or create new threads for request Tunable thread pool options Minimum size Maximum size Thread inactivity timeout Servers => Application Servers => server_name => Thread Pool
  • 37. TPV for Web container thread pool Default 50 is OK (预置值 为 50)
  • 38. Connection pool size(连接池大小)Goal – requests are not queued up waiting to access the database Tunable Connection pool options Minimum size Maximum size Connection inactivity timeout Resources => JDBC => Data Sources => data_source => Connection pool properties
  • 39. Before tuning 1 min/10 max10 conns are all used 33 waiting threads 8.2ms wait time
  • 40. After tuning 10 min/70 max0 waiting threads 0 wait time
  • 41. Performance benefit of tuned connection pool sizes
  • 42. Data source statement cache size (数据源 声明 缓存大小)Number of prepared JDBC statements that can be cached per connection Resources => JDBC => Data sources => data_source => WebSphere Application Server data source properties (WAS 数据源属性)
  • 43. After tuning from the default size 10 to 60 0 PrepStmt Cache Discard
  • 44. Performance benefit of increased data source statement cache size
  • 45. ORB pass by referenceEJB remote interface – Pass by value If client and EJB in same JVM Unnecessary pass by value – expensive Benefit from ORB pass by reference DayTrader WEB and EJB modules both are deployed to the same application server instance Servers => Application servers => server_name => Container services => ORB Service and select Pass by reference
  • 46. Performance benefit of ORB pass by reference
  • 47. Advanced tuning(高级调优)Servlet caching HTTP transport persistent connections Large page support (大页 支持) Disabling unused services (禁用 未使用的服务) Asynchronous messaging tuning (异步消息传递)
  • 48. Servlet caching(servlet高速缓存)Servers => Application servers => server_name => Web container settings => Web container cachespec.xml in WEB-INF
  • 49. Performance benefit of servlet caching
  • 50. HTTP transport persistent connectionsServe number of requests before close a HTTP connection SSL benefit most Servers => Application servers => server_name => Ports => View associated transports
  • 51. Large page supportUse large pages reduces pressure on shared hardware address translation resources Try 16MB for memory intensive application How to configures 1GB of 16MB large pages Enable: vmo -r -o lgpg_regions=64 -o lgpg_size=16777216 -o v_pinshm=1 Disable: vmo -o lgpg_regions=0 -o v_pinshm=0 Add –Xlp in Java command-line switches Servers => Application servers => server_name => Process definition => Java Virtual Machine
  • 52. Performance benefit of large pages (大页面的 表现效益)
  • 53. Disabling unused services - PMIMonitoring and Tuning => Performance Monitoring Infrastructure (PMI)
  • 54. Message store type(信息存储类型)File-based data store (default) (基于文件数据存储) Local Derby database data store (本地数据库数据存储) Remote database data store (远程数据库数据存储)
  • 55. Performance comparison of message store types (信息存储的性能比较类型)
  • 56. Moving transaction log and file store to a fast disk Servers => Application Servers => server_name => Container Services => Transaction Service
  • 57. Tuning summary (调优概要)JVM heap size Thread pool size Connection pool size Data source statement cache size ORB pass by reference Servlet caching Unlimited persistent HTTP connections Large page support Disabling PMI
  • 58. Overall performance improvements after tuning options applied
  • 59. AgendaOverview Tuning policies Tools PMI TPV Performance Advisor Request Metrics Case study DayTrader application Basic tuning Advanced tuning
  • 60. References(参考)Case study http://www.ibm.com/developerworks/websphere/techjournal/0909_blythe/0909_blythe.html WebSphere Application Server V7 InfoCenter http://publib.boulder.ibm.com/infocenter/wasinfo/v7r0/index.jsp
  • 61. Q & A