1. ESB设计与最佳实践陆正武
IBM Certificated SOA Architect
luzw@cn.ibm.com东风汽车有限公司
2. 议程ESB 、SOA与WebSphere的关系
什么时候需要用到ESB
ESB架构标准实践
ESB设计原则实践
IBM关于建立东风有限统一的ESB的建议方案
IBM ESB的独特优势
IBM ESB案例分享2
3. ESB、SOA和WebSphere的关系
4. As Patterns Have Evolved, So Has IBMFlexibility应用之间点对点的连接
实现简单、基本的信息交互和数据传递传统架构
基于消息传递的模式通过HUB模式实现应用之间的整合
很容易管理大量的连接和系统过渡架构
企业应用整合通过企业服务总线实现服务的整合集中和流程实现
借助标准的接口灵活地连接,实现真正的随需应变先进构架
面向服务体系架构
SOAIT构架的演变 The next stage of integration4
6. 关键概念SOA通过明确的定义和松散藕荷来提升系统间的弹性:
Service:服务
任何事情都可以是一个服务, 一个实现了唯一功能的自包容的实体
Service 交互
外部用户调用服务
服务可以彼此交互,调用操作和交换数据
Service交互可以是间接的
Service 编排
通过调用服务, 可以编排实现业务流程
Service 发现
一个注册的服务可以在构建时或者运行时被发现Service RequestorDiscovery AgencyDiscovery AgencyFindPublishInteractClientService ProviderService DescriptionServiceService RequestorDiscovery AgencyService RegistryFindPublishInteractClientService ProviderService DescriptionService6
7. Service InteractionEnterprise Service BusServiceServiceServiceServiceClient企业服务总线(ESB)概念经常用来表示服务互相交互的机制,它提供
了通信的基础架构
企业服务总线应当提供:
一致的、位置透明的、协议独立的手段来实现交互
一致的、位置透明的交互意味服务消费者和提供者之间可以是间接的
无论何种通信服务,应提供端到端的连通保证
7
8. 什么是Enterprise Service Bus?An Enterprise Service Bus is a core component of a Services Oriented Architecture …
它连接和集成企业和组织的IT infrastructure, across many
locations, 支持不同的transport services, and…
它可以自动路由消息并且提供系统级的功能,如:消息的格式自动
转换, and…
必须是基于标准的规范, and…
这些必须是安全的、可靠的,并且在高负荷的情况下可管理8
9. ESB应该有哪些服务?传输服务
安全、可靠的数据传输
永久性/非永久性
同步/异步
仲裁服务
路由
格式转换
事件服务
事件发现和发布
Publish / SubscribeShape = Protocol
Color = Data type
An Enterprise Service Bus (ESB) is a flexible connectivity infrastructure for integrating applications and services.
An ESB powers your SOA by reducing the number, size, and complexity of interfaces.9
12. SOA参考框架下WebSphere产品线IT 服务管理 业务创新和优化服务业务应用程序服务开发服务合作伙伴服务应用程序和信息资产连接性服务流程服务信息服务交互服务 Business ModelerWebSphere工具RationalBusiness
MonitorWebSphereTivoli工具PortalWebSphereWorkplaceIBMInformation IntegratorBI Server FoundationWebSphereWebSphereWebSphereWebSphere
Application Server
Partner GatewayMQWebSphereMessage BrokerWebSphereTivoliIntelligent OrchestratorTivoli MonitoringTivoli Access ManagerInformation IntegratorWebSphereBusiness Integration
AdaptersWebSphereWebSphereESBWebSphereProcess ServerIntegration DeveloperWebSphereWebSphereDataPower12
13. 什么时候需要用 ESB
14. 什么时候需要用 ESBESB 网关 – recommendation DataPower SOA Appliance – for any external services to protect against XML threats
中介 between non-standards based systems and standards based systems
支持动态更改提供者 (routing, replacement, load balancing, protocol changes)
Including versioning
Minimize changes to requestors
支持请求者与提供者之间协议和数据格式不同时的交互
Adding new requestors who are not an exact match with provider
Common security, management, monitoring enforcement
14
15. ESB架构模式核心原则ESB inter-connects requester and provider
Interactions are decoupled
Supports key SOA principle – separation of concerns
ESB provides Service Virtualization of
Identity via routing
Interaction via conversion
Interface via transformation
ESB also enables Aspect Oriented Connectivity
Security
Management
Logging
Auditing
…
Described in a whitepaper by Greg Flurry
http://www-128.ibm.com/developerworks/architecture/library/ar-esbpat1/Service RequestorService ProviderESB15
16. ESB架构实践
17. 1. 确定参考架构ESB涉及多平台,多项目
Usually spearheaded by a corporate Architecture team or Infrastructure team
Needs participation and agreement from Networking, Security, Application Development, and Operations
A Reference Architecture is a key document and process for securing this agreement across all stakeholders
核心架构、设计和管控决策
可选的技术路线和设计模式
提供常见场景和使用实践17
18. 定义接口和连接标准ESB功能全面,大多数企业不会使用到所有功能
You need to decide what connectivity options to allow both internally and externally
确定总线上实现哪些功能及gateways 和 adapters
Gateways are access points for bridging on and off the bus
Can be used as policy enforcement points
Adapters are mechanisms for bringing information onto or off of the bus
确定采用哪种标准路线
Emphasize industry standards for external (partner) communication
确定所支持的常见数据表示
Canonical message formats allow for shared transformation mediations
18
19. Message Models :XMLMediation FlowsCommunication ProtocolsEnterprise Service BusInteraction PatternsMediation PatternsMeta modelsExpanded view of the ESB19
20. 以ESB为中心的逻辑模型Outside ESB
Business Logic (Business Services)
ESB does contain integration logic or connectivity logic
Criteria: semantics versus syntax; aspects
Loosely coupled to ESB
Security and Management
Policy Decision Point outside the ESB
ESB can be Policy Enforcement PointTightly coupled to ESB
Service Registry
Registry a Policy Decision Point for ESB
ESB a Policy Enforcement Point for Registry
But, Registry has a broader scope in SOA
Tooling required for ESB
Development
Administration
Configures ESB via Service RegistryESBManagement ServicesSecurityManagementApplication ServicesInformationProcessInteractionAccessBusiness AppPartnerRegistryDevelopment Services20
21. 2. 定义公共术语表ESB没有统一的术语
Vendors and clients are variable in their definition of what is part of the ESB and terminology
IBM has varying terminology in our ESB products
Patterns work has its own terminology
构建ESB/SOA时非常重要的一点就是建立统一的术语表Generic TermMediation Flow (or simply Mediation)Mediation PatternWMBMessage FlowNodeWESBMediation ModuleMediation primitiveDataPowerPolicyPolicy Step21
22. 使用规范还是标准?A specification outlines APIs/data/formats from one or more vendors working together
Proposed by vendors to address a gap
Often supported by some vendor’s software
Usually submitted to a standards body
More companies get involved, specification changes, sometimes merged with competing specifications
Goes through multiple drafts and reviews
Standard is agreed to by the standards body’s committee members and is declared a ‘version x’ standard
Standards bodies vary by process, rigor, respect and adoption rate
And specific standards vary wrt to adoption
22
23. 3. Apply criteria on whether to use a standard or specification 是否很好地描述了我们的需求?
标准是否覆盖了我们的需求?
是否需要在不同平台上的互操作性和可移植性?
采用的到底是标准还是规范,标准和规范的成熟度如何?
是否支持所有的供应商平台,版本是否具有连续性?
采用的标准是否支持WS-I? 我们的供应商是否也支持?
是否需要采用最新版本的标准?
如果供应商系统不支持现有标准,只支持早期的规范,如何移植?
是否可以增加一层标准的API,以保护你的代码?
采用的标准和规范是否比我们所需要的更复杂?23
24. 4. 从企业基础架构中分离出业务逻辑基本设计原则:需要分离考虑:
Business logic does not belong in mediations
ESB 架构于业务逻辑之上 - the business logic may put information into headers
Example: business logic determines whether a customer deserves priority treatment, sets a flag in the header, the ESB routes to highest performing provider
分离业务逻辑和应用程序逻辑
Use BPEL and choreography for that instead24
25. 5. 采用基于模式的研究方法Enterprise Integration Patterns (Hohpe) has a set of 60+ patterns that cover common messaging and EI situations
Includes an abstract notation for representing the patterns
Using this notation allows you to decide on the content and flow of the mediations without having implementation considerations drive the process
帮助实现多个ESB产品之间耦合的配置
方便查找应用程序之间的公共部分
在大多数ESB工具中容易实现 (WMB, WESB, DataPower)
Use the IBM Patterns for e-Business to map out your ESB topologies25
27. The SOA/ESB Design Pattern (P4eB)Demilitarized ZoneWeb Services Gateway
Partner ZoneEnterprise Secure ZoneEnterprise Service BusEnterprise Service Bus Gateway
Back-end Resource Gateway
Domain FirewallInter-Enterprise ZoneProtocol FirewallNetwork Infrastructure27
28. 6. 确定服务提供者查找方法论– ESB 需要提供者的源头An ESB must send the message to one or more provider endpoints
在可选的提供者之间作出选择
Options for managing endpoint lookup
Not recommended
No endpoint lookup (fixed addresses)
Change endpoint administratively on the ESB servers
Provider must keep track of all ESB servers which can call it
IP spraying or load balancing between ESB and providers
Limited to IP, providers maybe outside of your domain
Endpoint(s) lookup from a registry
使用Cache,而不是直接向注册库
Providers update the registry with new endpoints
Separates ESB from the providers
允许无缝增加新的providers 和ESB服务器
28
29. (6) Message is transformed and routed to the selected endpoint.Message(1) A Message is receivedESB
Mediation(2) Invokes a selection mediation.PublishFindEnrichGovernManageWebSphere Service Registry and Repository(3) Retrieve requestor information(4) Retrieves candidate providers informationMessage(5) Executes matching algorithm to identify the provider service for requestor serviceMessageServiceDynamic selectionWSRR support
WESB 6.0.2 mediation primitive
Port Type match
WMB SupportPac
Port Type match
General metadata retrieval
DP 3.6.0.8 and higher firmware7. 使用 Service 注册表 29
31. Services Versioning ApproachesApproachProsConsSchema VersioningEnforceable with schema validationRequires careful planningMultiple VersionsSimple design and implementationExpensive to implement ESB MediationWorks even when schema evolution not plannedPerformance,
Complex implementation31
32. 9. 设置端到端的 QOS指导方针在SOA中,性能方面的考虑比其他非SOA的应用复杂得多
Need to include mediations and transports in performance budgets
More shared resources to consider
需要了解每个service的负载及应用程序中介的负担
对SOA流程端到端的监控计划
采用可选的技术,如DP 和 WebSphere XD 来管理 QOS32
33. 10. 设立公共安全策略ESB是共享资源,安全策略更为重要
This is a least-common denominator from which you can expand
Security must be holistic
Each enterprise has its own balance of risk and countermeasures but ultimately must support the business model for information protection & user access
定义端到端的安全模型
Document existing security practices, policies and requirements
Perform a risk assessment to document vulnerabilities and countermeasures to address known risks
What are the legal issues involved?
Are there requirements where policies are(/should be) being enforced?
Is a manual procedure (code review) for application development the control point?
确定安全策略及策略控制点
33
34. 11. Determine Ownership of the ESB The question of who owns the ESB(s) often nearly stops many ESB projects
网络组?
核心架构组?
项目组管理 ESBs?
大多数案例中,中央集权的管理是成功的
在ESB中只部分需要共享的WEB资源
If centralization is impossible, then a central team should control federation34
35. ESB设计原则实践
36. 1. 确定规范或不规范消息模型在ESB中建议采用一致规范的消息模型.
If any requester or provider does not understand the canonical message ([Hohpe]), then the canonical message is transformed into the message format of the requester or provider.
This transformation may be done by ESB or one the requester or provider.
如果没有规范的消息模型,ESB就成了点对点的集成
Which increases cost to add new requestors and providers. 36
37. When do you adopt a standard or a specification?Industry Standards and XML Schema issuesESB 采用工业标准的规范格式
在需要集成供应商和客户的系统时更需要采用工业标准
不同行业有自己的行业标准
Some popular ones HL7 for healthcare, ACORD for insurance
有些客户需要扩展某些标准
Many industry schemas leverage XML Schema elements that are not (yet) well supported by Web service environments/tools
Most common examples are and xsd:choice>
Vendor support for these complex schema elements is varied in some cases proprietary or not fully supported
Try to avoid those if you can
JAX-RPC has limited schema support - JAX-WS 2.0 / JAXB 2.0 will support full set of schema elements
In our products
JAX-RPC supported by WAS 5.0.2 – 6.1 so limited support
WAS 6.0 and above, you can plug in JAX-B and/or custom serializers
Web services feature pack will support JAX-WS and JAX-B
WebSphere ESB and WPS have limited support
37
38. 利用Adapters集成现有系统An ESB must allow access to existing Enterprise Information Systems
Adapters typically used, and may or may not be part of the ESB
Technology 技术适配器
Application 应用适配器
Legacy 遗留系统适配器
适配器与ESB的位置关系:
Outside of the ESB, and inside the EIS domain
Outside the ESB, and the EIS domain
On the boundary of the ESB
Inside the ESB
Adapters两个关键技术点
Communication protocol
Message format38
39. 2. 确定哪个应用程序需要适应总线Who is responsible for building adapters on to the bus is a governance issue. It is frequently driven by ownership, power, and funding considerations.
It may be influenced by the technical considerations
Options:
ESB declares canonical messages and all requestors and providers adapt to the canonical format.
ESB adapts all providers (or a limited set) to a canonical format. Requestors adapt to canonical format.
ESB adapts requestors (usually legacy requestor) to canonical or provider specific format.
ESB adapts directly from a provider format to a specific requestor format (basically point to point).39
40. 3. 仅当需要的时候才用可靠性和事务处理功能 QoS includes not only performance but reliability measures as well
Use reliable delivery sparingly
Relatively high performance penalty
Limited to some transports or standards
少用事务
利用 services 而不是本地调用
40
41. 4. 将基本信息放在消息头中,而不是消息体中Place infrastructure logic such as security, logging levels, client application identify in a header which is separate from the body of a message
ESBs are designed to support separation of infrastructure processing from service logic and are frequently optimized to use headers.
It's better to use headers which are protocol independent to allow header information to seamlessly flow across the various protocols the bus supports.
No header mapping is required as the bus converts to a new protocol
Included in SOAP is separation of the header from the body
When designing your own message formats, have a header section and a body section41
42. 5. ESB联邦设计基本上每个大型ESB项目都会遇到ESB联邦
Multiple products used for ESB functionality
Multiple sites dictate multiple product installations
Organizational, Corporate or political boundaries may dictate the boundary of an ESB
ESB联邦最佳解决方案是采用gateway
Routing requests to the appropriate ESB (local or remote)
Handling special security requirements across boundaries
Providing logging and auditing requirements across boundaries
Dealing with technology differences in transport or format42
43. ESB 联邦WESBWMBHTTP Backbone
WS-SecurityMQ Backbone
Transport SecurityXI50 Gateway站点 1站点243
44. IBM关于建立东风有限统一的ESB的建议方案
45. IBM ESB简介标准的企业服务总线:WebSphere ESB
高级的企业服务总线:WebSphere Message Broker
企业服务总线Gateway:DataPower
45
47. WebSphere Application ServerIncreased capabilities / automationBusiness ProcessESBA world-class J2EE foundation providing industry-leading levels of availability, scalability, and performance.
WebSphere Process ServerFor customers who want a higher level solution to design, automate and manage composite applications and operational business processes. Built on WebSphere ESB.WebSphere ESBA new product that delivers an Enterprise Service Bus. Provides Web Services connectivity and data transformation. Built on WebSphere Application Server.SubsetSupersetWebSphere ESB的位置47
48. WebSphere ESB and WebSphere Message BrokerESB:
WebSphere ESBAdvanced ESB:
WebSphere Message BrokerCustomers face a range of ESB requirements. As a result, any given project might require an ESB or an Advanced ESB… OR BOTH.Web Services connectivity
and data transformationWeb ServicesJMSWebSphere MQHTTPXMLWebSphere AdaptersJMSTIBCO Rendezvous®
Web ServicesWeblogic JMS®
WebSphere MQTIBCO EMS JMS®
HTTPSonicMQ JMS®Biztalk®
MQeTuxedo®
Real-time IPFTPMulticastMQTTXMLCOBOL
CopybookEDI-X.12ACORDEDI-FACTebXMLAL3HIPAAHL7SWIFTFIXCustom FormatsWord/Excel/PDFPlus the following:WebSphere
AdaptersUniversal connectivity and data transformation核心版高级版48
49. SOA 利器 —— WebSphere DataPower IBM 和 DataPower 利用专用的耐久性SOA设备 – 为SOA实施提供优异的性能与硬件安全性 –重新定义了中间件的界限,扩大了SOA的基础。简化 SOA
专用的 SOA 设备,部署、管理及维护更加简便,降低成本;
保护 SOA
集中化管理、丰富的安全策略,更高的可靠性,有力保障系统安全;
加速 SOA
无与伦比的 XML 处理能力,大幅提高系统处理速度和吞吐量; 49
50. DataPower 典型应用场景保护对外的 Web 服务
支持合作伙伴、客户安全地访问记录的后端系统
传统系统集成
把大型机或传统应用连接到 Web 服务SOA
HUB 式的中介服务
在应用和 Web 服务间有效地转换、路由、记录消息
加速动态的 Web 站点
加快 XML-HTML 渲染,用于动态内容生成50
51. 东风汽车有限公司ESB建议方案企业组ESB
部门/应用级ESB
ESB Gateway51
52. Mediation Chain格式转换服务路由企业级ESB典型方案Enterprise Service Bus(MB)交互服务Web页面内部流程服务Web页面DMS召回系统SAP…………OAAdapterSOAP/JMS/IIOPSOAP/JMS/IIOPSOAP/JMS/IIOPDataPowerSOAP/JMS/IIOPSOAP/JMS/IIOP新车销售系统SOAP/JMS/IIOPMQMQEnterprise Service Bus(MB/WESB)SCM系统WEB展现52
53. WPS Cluster
开发与部署架构WebSphere Message BrokerBroker DomainBrokerBrokerWebSphere Message BrokerBroker DomainBrokerBrokerESB Cluster
User
Appv6 Config ManagerConfig Manager ProxyUser
AppConfig Manager Proxy开发AdapterMQJMSHTTP/SOAPWebSphere Process ServerNode AgentWPSWPSWebSphere Process ServerNode AgentWPSWPS53