C# 5.0 和.NET 4.5.1 高级编程


ffirs.indd 3 30-01-2014 20:54:07 Professional C# 5.0 and .NET 4.5.1 Current Author Team Christian Nagel Jay Glynn Morgan Skinner Authors On Previous Editions Bill Evjen Karli Watson ffirs.indd 1 30-01-2014 20:54:07 Professional C# 5.0 and .NET 4.5.1 Published by John Wiley & Sons, Inc. 10475 Crosspoint Boulevard Indianapolis, IN 46256 www.wiley.com Copyright © 2014 by John Wiley & Sons, Inc., Indianapolis, Indiana Published simultaneously in Canada 978-1-118-83303-2 978-1-118-83294-3 (ebk) 978-1-118-83298-1 (ebk) Manufactured in the United States of America 10 9 8 7 6 5 4 3 2 1 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions. Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or pro- motional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the pub- lisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or Web site may provide or recommendations it may make. Further, readers should be aware that Internet Web sites listed in this work may have changed or disappeared between when this work was written and when it is read. For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with stan- dard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com. Library of Congress Control Number: 2013958290 Trademarks: Wiley, the Wiley logo, Wrox, the Wrox logo, Programmer to Programmer, and related trade dress are trade- marks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc., is not associated with any product or vendor mentioned in this book. ffirs.indd 2 30-01-2014 20:54:07 To my family – Angela, Stephanie, and Matthias – I love you all! —Christian Nagel This work is dedicated to my wife and son. They are my world. —Jay Glynn Love is as strong as death; Many waters cannot quench love, Neither can the floods drown it. —Morgan Skinner ffirs.indd 3 30-01-2014 20:54:07 Acquisitions Editor Mary James Project Editor Charlotte Kughen Technical Editors Don Reamey George Evjen Production Editor Christine Mugnolo Editorial Manager Mary Beth Wakefield Freelancer Editorial Manager Rosemarie Graham Associate Director of Marketing David Mayhew Marketing Manager Ashley Zurcher Business Manager Amy Knies Vice President and Executive Group Publisher Richard Swadley Associate Publisher Jim Minatel Project Coordinator, Cover Katie Crocker Proofreader Sarah Kaikini, Word One, New York Indexer Johnna VanHoose Dinse Cover Designer Wiley Cover Image © Henrik5000/istockphoto.com Credits ffirs.indd 4 30-01-2014 20:54:07 About the Authors Christian Nagel  is a Microsoft Regional Director and Microsoft MVP, an associate of thinktecture, and founder of CN innovation. A software architect and developer, he offers training and consulting on how to develop solutions using the Microsoft platform. He draws on more than 25 years of software development experience. Christian started his computing career with PDP 11 and VAX/VMS systems, covering a variety of languages and platforms. Since 2000, when .NET was just a technology preview, he has been working with various .NET technologies to build .NET solutions. Currently, he mainly coaches the development of Windows Store apps accessing Windows Azure services. With his profound knowledge of Microsoft technologies, he has written numerous books, and is certified as a Microsoft Certified Trainer (MCT) and Solutions Developer (MCSD). Christian speaks at international conferences such as TechEd, Basta!, and TechDays, and he founded INETA Europe to support .NET user groups. You can contact Christian via his website www.cninnovation.com, read his blog at blogs.thinktecture.com/cnagel, and follow his tweets at @christiannagel. Jay Glynn  started writing software more than 20 years ago, writing applications for the PICK operating system using PICK basic. Since then, he has created software using Paradox PAL and Object PAL, Delphi, VBA, Visual Basic, C, Java, and of course C#. He currently works for VGT as a software engineer writing server-based software. Morgan Skinner  began his computing career at a young age on the Sinclair ZX80 at school, where he was underwhelmed by some code a teacher had written and so began programming in assembly language. Since then he has used a wide variety of languages and platforms, including VAX Macro Assembler, Pascal, Modula2, Smalltalk, X86 assembly language, PowerBuilder, C/C++, VB, and currently C#. He’s been ­programming in .NET since the PDC release in 2000, and liked it so much he joined Microsoft in 2001. He’s now an independent consultant. Business Manager Amy Knies Vice President and Executive Group Publisher Richard Swadley Associate Publisher Jim Minatel Project Coordinator, Cover Katie Crocker Proofreader Sarah Kaikini, Word One, New York Indexer Johnna VanHoose Dinse Cover Designer Wiley Cover Image © Henrik5000/istockphoto.com Credits ffirs.indd 5 30-01-2014 20:54:07 About the TECHNICAL Editors Don Reamey  is an architect/principal engineer for TIBCO Software working on TIBCO Spotfire business intelligence analytics software. Prior to TIBCO Don spent 12 years with Microsoft as a software develop- ment engineer working on SharePoint, SharePoint Online and InfoPath Forms Service. Don has also spent 10 years writing software in the financial service industry for capital markets. George Evjen  is the director of development for ArchitectNow, a St. Louis-based consulting company specializing in custom client application architecture, design, and development, with clients ranging from small technology start-ups to global enterprises. Prior to his involvement in the software industry, George spent more than a dozen years coaching men’s basketball at all levels of the collegiate ranks. As a moti- vational leader with an infectious positive outlook in nearly all situations, he is the ideal person to take the lead directly for many of ArchitectNow’s largest projects and clients. Not only does he work as a lead developer, but he also manages most of the coordination between ArchitectNow and the company’s external contractors and resources. George has extensive experience and expertise in all of Microsoft’s web-based and XAML-based ­technologies, as well as the newest web frameworks available. His specialties include enterprise-level WPF, Silverlight, and Windows 8 projects, as well as ASP.NET MVC business application development. He speaks to groups and at conferences around the region on topics of motivational leadership, project ­management, and organization. You can find additional information on George and ArchitectNow’s capabilities at http://www.architectnow.net. ffirs.indd 6 30-01-2014 20:54:07 Acknowledgments I would like to thank  Charlotte Kughen for making this text more readable; Mary James; and Jim Minatel; and everyone else at Wiley who helped to get another edition of this great book published. I would also like to thank my wife and children for supporting my writing. You’re my inspiration.  — Christian Nagel I want to thank  my wife and son for putting up with the time and frustrations of working on a project like this. I also want to thank all the dedicated people at Wiley for getting this book out the door.  — Jay Glynn ffirs.indd 7 30-01-2014 20:54:07 ffirs.indd 8 30-01-2014 20:54:07 Contents Introduction xxiii Part I: The C# Language Chapter 1: .NET Architecture 3 The Relationship of C# to .NET 3 The Common Language Runtime 4 A Closer Look at Intermediate Language 7 Assemblies 14 .NET Framework Classes 16 Namespaces 17 Creating .NET Applications Using C# 18 The Role of C# in the .NET Enterprise Architecture 21 Summary 22 Chapter 2: Core C# 23 Fundamental C# 24 Your First C# Program 24 Variables 27 Predefined Data Types 31 Flow Control 37 Enumerations 43 Namespaces 45 The Main() Method 47 More on Compiling C# Files 49 Console I/O 50 Using Comments 52 The C# Preprocessor Directives 54 C# Programming Guidelines 57 Summary 63 Chapter 3: Objects and Types 65 Creating and Using Classes 65 Classes and Structs 66 Classes 66 Anonymous Types 79 ftoc.indd 9 30-01-2014 20:59:59 x CONTENTS Structs 80 Weak References 82 Partial Classes 83 Static Classes 85 The Object Class 85 Extension Methods 87 Summary 88 Chapter 4: Inheritance 89 Inheritance 89 Types of Inheritance 89 Implementation Inheritance 90 Modifiers 99 Interfaces 100 Summary 105 Chapter 5: Generics 107 Generics Overview 107 Creating Generic Classes 110 Generics Features 114 Generic Interfaces 118 Generic Structs 122 Generic Methods 124 Summary 128 Chapter 6: Arrays and Tuples 129 Multiple Objects of the Same and Different Types 129 Simple Arrays 130 Multidimensional Arrays 132 Jagged Arrays 133 Array Class 134 Arrays as Parameters 139 Enumerations 140 Tuples 146 Structural Comparison 147 Summary 149 Chapter 7: Operators and Casts 151 Operators and Casts 151 Operators 151 ftoc.indd 10 30-01-2014 20:59:59 xi CONTENTS Type Safety 157 Comparing Objects for Equality 162 Operator Overloading 163 User-Defined Casts 172 Summary 181 Chapter 8: Delegates, Lambdas, and Events 183 Referencing Methods 183 Delegates 184 Lambda Expressions 198 Events 201 Summary 208 Chapter 9: Strings and Regular Expressions 209 Examining System.String 210 Regular Expressions 221 Summary 228 Chapter 10: Collections 229 Overview 230 Collection Interfaces and Types 230 Lists 231 Queues 241 Stacks 245 Linked Lists 246 Sorted List 251 Dictionaries 252 Sets 259 Observable Collections 260 Bit Arrays 262 Immutable Collections 266 Concurrent Collections 268 Performance 275 Summary 277 Chapter 11: Language Integrated Query 279 LINQ Overview 279 Standard Query Operators 287 Parallel LINQ 305 Expression Trees 307 ftoc.indd 11 30-01-2014 20:59:59 xii CONTENTS LINQ Providers 310 Summary 310 Chapter 12: Dynamic Language Extensions 313 Dynamic Language Runtime 313 The Dynamic Type 314 Hosting the DLR ScriptRuntime 318 DynamicObject and ExpandoObject 321 Summary 324 Chapter 13: Asynchronous Programming 325 Why Asynchronous Programming Is Important 325 Asynchronous Patterns 326 Foundation of Asynchronous Programming 338 Error Handling 341 Cancellation 344 Summary 346 Chapter 14: Memory Management and Pointers 347 Memory Management 347 Memory Management Under the Hood 348 Freeing Unmanaged Resources 353 Unsafe Code 358 Summary 372 Chapter 15: Reflection 373 Manipulating and Inspecting Code at Runtime 373 Custom Attributes 374 Using Reflection 380 Summary 389 Chapter 16: Errors and Exceptions 391 Introduction 391 Exception Classes 392 Catching Exceptions 393 User-Defined Exception Classes 402 Caller Information 409 Summary 411 ftoc.indd 12 30-01-2014 20:59:59 xiii CONTENTS Part ii: Visual Studio Chapter 17: Visual Studio 2013 415 Working with Visual Studio 2013 415 Creating a Project 420 Exploring and Coding a Project 425 Building a Project 437 Debugging Your Code 441 Refactoring Tools 447 Architecture Tools 448 Analyzing Applications 451 Unit Tests 457 Windows Store Apps, WCF, WF, and More 463 Summary 467 Chapter 18: Deployment 469 Deployment as Part of the Application Life Cycle 469 Planning for Deployment 470 Traditional Deployment 471 ClickOnce 473 Web Deployment 479 Windows Store Apps 481 Summary 486 Part iiI: Foundation Chapter 19: Assemblies 489 What are Assemblies? 489 Application Domains 499 Shared Assemblies 503 Configuring .NET Applications 510 Versioning 513 Sharing Assemblies Between Different Technologies 517 Summary 520 Chapter 20: Diagnostics 521 Diagnostics Overview 521 Code Contracts 522 Tracing 528 ftoc.indd 13 30-01-2014 21:00:00 xiv CONTENTS Event Logging 540 Performance Monitoring 548 Summary 554 Chapter 21: Tasks, Threads, and Synchronization 555 Overview 556 Parallel Class 557 Tasks 561 Cancellation Framework 566 Thread Pools 569 The Thread Class 570 Threading Issues 574 Synchronization 579 Timers 597 Data Flow 598 Summary 602 Chapter 22: Security 605 Introduction 605 Authentication and Authorization 606 Encryption 614 Access Control to Resources 621 Code Access Security 623 Distributing Code Using Certificates 629 Summary 630 Chapter 23: Interop 631 .NET and COM 631 Using a COM Component from a .NET Client 638 Using a .NET Component from a COM Client 649 Platform Invoke 659 Summary 663 Chapter 24: Manipulating Files and the Registry 665 File and the Registry 665 Managing the File System 666 Moving, Copying, and Deleting Files 674 Reading and Writing to Files 677 Mapped Memory Files 692 Reading Drive Information 693 ftoc.indd 14 30-01-2014 21:00:00 xv CONTENTS File Security 695 Reading and Writing to the Registry 699 Reading and Writing to Isolated Storage 704 Summary 707 Chapter 25: Transactions 709 Introduction 709 Overview 710 Database and Entity Classes 712 Traditional Transactions 713 System.Transactions 716 Dependent Transactions 721 Isolation Level 729 Custom Resource Managers 731 File System Transactions 737 Summary 740 Chapter 26: Networking 741 Networking 741 The HttpClient Class 742 Displaying Output as an HTML Page 746 Utility Classes 756 Lower-Level Protocols 759 Summary 771 Chapter 27: Windows Services 773 What Is a Windows Service? 773 Windows Services Architecture 775 Creating a Windows Service Program 777 Monitoring and Controlling Windows Services 793 Troubleshooting and Event Logging 802 Summary 803 Chapter 28: Localization 805 Global Markets 805 Namespace System.Globalization 806 Resources 817 Windows Forms Localization Using Visual Studio 823 Localization with ASP.NET Web Forms 830 Localization with WPF 832 ftoc.indd 15 30-01-2014 21:00:00 xvi CONTENTS A Custom Resource Reader 837 Creating Custom Cultures 840 Localization with Windows Store Apps 842 Summary 845 Chapter 29: Core XAML 847 Uses of XAML 847 XAML Foundation 848 Dependency Properties 853 Bubbling and Tunneling Events 856 Attached Properties 859 Markup Extensions 861 Reading and Writing XAML 863 Summary 864 Chapter 30: Managed Extensibility Framework 865 Introduction 865 MEF Architecture 866 Defining Contracts 873 Exporting Parts 875 Importing Parts 884 Containers and Export Providers 889 Catalogs 892 Summary 893 Chapter 31: Windows Runtime 895 Overview 895 Windows Runtime Components 902 Windows Store Apps 905 The Life Cycle of Applications 907 Application Settings 913 Summary 916 Part iv: Data Chapter 32: Core ADO.NET 919 ADO.NET Overview 919 Using Database Connections 922 Commands 927 ftoc.indd 16 30-01-2014 21:00:00 xvii CONTENTS Fast Data Access: The Data Reader 934 Asynchronous Data Access: Using Task and Await 936 Managing Data and Relationships: The DataSet Class 938 XML Schemas: Generating Code with XSD 948 Populating a DataSet 953 Persisting DataSet Changes 955 Working with ADO.NET 958 Summary 963 Chapter 33: ADO.NET Entity Framework 965 Programming with the Entity Framework 965 Entity Framework Mapping 967 Entities 972 Data Context 973 Relationships 975 Querying Data 980 Writing Data to the Database 982 Using the Code First Programming Model 987 Summary 995 Chapter 34: Manipulating XML 997 XML 997 XML Standards Support in .NET 998 Introducing the System.Xml Namespace 998 Using System.Xml Classes 999 Reading and Writing Streamed XML 1000 Using the DOM in .NET 1007 Using XPathNavigators 1011 XML and ADO.NET 1020 Serializing Objects in XML 1027 LINQ to XML and .NET 1036 Working with Different XML Objects 1036 Using LINQ to Query XML Documents 1042 More Query Techniques for XML Documents 1045 Summary 1048 Part v: Presentation Chapter 35: Core WPF 1051 Understanding WPF 1052 ftoc.indd 17 30-01-2014 21:00:00 xviii CONTENTS Shapes 1055 Geometry 1056 Transformation 1058 Brushes 1060 Controls 1063 Layout 1068 Styles and Resources 1071 Triggers 1077 Templates 1080 Animations 1089 Visual State Manager 1095 3-D 1098 Summary 1102 Chapter 36: Business Applications with WPF 1103 Introduction 1103 Menu and Ribbon Controls 1104 Commanding 1107 Data Binding 1109 TreeView 1139 DataGrid 1143 Summary 1154 Chapter 37: Creating Documents with WPF 1155 Introduction 1155 Text Elements 1156 Flow Documents 1164 Fixed Documents 1168 XPS Documents 1171 Printing 1173 Summary 1175 Chapter 38: Windows Store Apps: User Interface 1177 Overview 1177 Microsoft Modern Design 1178 Sample Application Core Functionality 1180 App Bars 1187 Launching and Navigation 1188 Layout Changes 1190 Storage 1195 ftoc.indd 18 30-01-2014 21:00:00 xix CONTENTS Pickers 1201 Live Tiles 1202 Summary 1204 Chapter 39: Windows Store Apps: Contracts and Devices 1205 Overview 1205 Searching 1206 Sharing Contract 1208 Camera 1212 Geolocation 1213 Sensors 1216 Summary 1221 Chapter 40: Core ASP.NET 1223 .NET Frameworks for Web Applications 1223 Web Technologies 1225 Hosting and Configuration 1226 Handlers and Modules 1229 Global Application Class 1233 Request and Response 1234 State Management 1236 ASP.NET Identity System 1247 Summary 1251 Chapter 41: ASP.NET Web Forms 1253 Overview 1253 ASPX Page Model 1254 Master Pages 1263 Navigation 1267 Validating User Input 1268 Accessing Data 1271 Security 1280 Ajax 1283 Summary 1296 Chapter 42: ASP.NET MVC 1297 ASP.NET MVC Overview 1297 Defining Routes 1299 ftoc.indd 19 30-01-2014 21:00:00 xx CONTENTS Creating Controllers 1300 Creating Views 1304 Submitting Data from the Client 1314 HTML Helpers 1318 Creating a Data-Driven Application 1323 Action Filters 1331 Authentication and Authorization 1332 Summary 1336 Part vI: Communication Chapter 43: Windows Communication Foundation 1339 WCF Overview 1339 Creating a Simple Service and Client 1342 Contracts 1354 Service Behaviors 1358 Binding 1362 Hosting 1368 Clients 1370 Duplex Communication 1372 Routing 1374 Summary 1379 Chapter 44: ASP.NET Web API 1381 Overview 1381 Creating Services 1382 Creating a .NET Client 1385 Web API Routing and Actions 1388 Using OData 1391 Security with the Web API 1400 Self-Hosting 1405 Summary 1406 Chapter 45: Windows Workflow Foundation 1407 A Workflow Overview 1407 Hello World 1408 Activities 1409 Custom Activities 1413 Workflows 1419 Summary 1432 ftoc.indd 20 30-01-2014 21:00:00 xxi CONTENTS Chapter 46: Peer-to-Peer Networking 1433 Peer-to-Peer Networking Overview 1433 Peer Name Resolution Protocol (PNRP) 1437 Building P2P Applications 1439 Summary 1445 Chapter 47: Message Queuing 1447 Overview 1448 Message Queuing Products 1450 Message Queuing Architecture 1451 Message Queuing Administrative Tools 1452 Programming Message Queuing 1453 Course Order Application 1460 Receiving Results 1470 Transactional Queues 1471 Message Queuing with WCF 1472 Message Queue Installation 1478 Summary 1478 Index 1479 ftoc.indd 21 30-01-2014 21:00:00 flast.indd 22 30-01-2014 20:59:28 Introduction If you were to describe the C# language  and its associated environment, the .NET Framework, as the most significant technology for developers available, you would not be exaggerating. .NET is designed to provide an environment within which you can develop almost any application to run on Windows, whereas C# is a programming language designed specifically to work with the .NET Framework. By using C#, you can, for example, write a dynamic web page, a Windows Presentation Foundation application, an XML web service, a component of a distributed application, a database access component, a classic Windows desktop application, or even a new smart client application that enables online and offline capabilities. This book covers the .NET Framework 4.5.1. If you code using any of the prior versions, there may be sections of the book that will not work for you. This book notifies you of items that are new and specific to the .NET Framework 4.5 and 4.5.1. Don’t be fooled by the .NET label in the Framework’s name and think that this is a purely Internet- focused framework. The .NET bit in the name is there to emphasize Microsoft’s belief that distributed applications, in which the processing is distributed between client and server, are the way forward. You must also understand that C# is not just a language for writing Internet or network-aware applications. It provides a means for you to code almost any type of software or component that you need to write for the Windows platform. Between them, C# and .NET have revolutionized the way that developers write their programs and have made programming on Windows much easier than it has ever been before. So what’s the big deal about .NET and C#? The Significance of .NET and C# To understand the significance of .NET, you must consider the nature of many of the Windows technologies that have appeared in the past 20 years. Although they may look quite different on the surface, all the Windows operating systems from Windows NT 3.1 (introduced in 1993) through Windows 8.1 and Windows Server 2012 R2 have the same familiar Windows API for Windows desktop and server applications at their core. Progressing through new versions of Windows, huge numbers of new functions have been added to the API, but this has been a process to evolve and extend the API rather than replace it. With Windows 8, the main API of the operating system gets a replacement with Windows Runtime. However, this runtime is still partly based on the familiar Windows API. The same can be said for many of the technologies and frameworks used to develop software for Windows. For example, Component Object Model (COM) originated as Object Linking and Embedding (OLE). Originally, it was largely a means by which different types of Office documents could be linked so that you could place a small Excel spreadsheet in your Word document, for example. From that it evolved into COM, Distributed COM (DCOM), and eventually COM+ — a sophisticated technology that formed the basis of the way almost all components communicated, as well as implementing transactions, messaging services, and object pooling. Microsoft chose this evolutionary approach to software for the obvious reason that it is concerned with backward compatibility. Over the years, a huge base of third-party software has been written for Windows, and Windows would not have enjoyed the success it has had if every time Microsoft introduced a new technology it broke the existing code base! flast.indd 23 30-01-2014 20:59:29 xxiv introduction Although backward compatibility has been a crucial feature of Windows technologies and one of the strengths of the Windows platform, it does have a big disadvantage. Every time some technology evolves and adds new features, it ends up a bit more complicated than it was before. It was clear that something had to change. Microsoft could not go on forever extending the same development tools and languages, always making them more and more complex to satisfy the conflicting demands of keeping up with the newest hardware and maintaining backward compatibility with what was around when Windows first became popular in the early 1990s. There comes a point in which you must start with a clean slate if you want a simple yet sophisticated set of languages, environments, and developer tools, which makes it easy for developers to write state-of-the-art software. This fresh start is what C# and .NET were all about in the first incarnation. Roughly speaking, .NET is a framework — an API — for programming on the Windows platform. Along with the .NET Framework, C# is a language that has been designed from scratch to work with .NET, as well as to take advantage of all the progress in developer environments and in your understanding of object-oriented programming principles that have taken place over the past 25 years. Before continuing, you must understand that backward compatibility has not been lost in the process. Existing programs continue to work, and .NET was designed with the capability to work with existing software. Presently, communication between software components on Windows takes place almost entirely using COM. Taking this into account, the .NET Framework does have the capability to provide wrappers around existing COM components so that .NET components can talk to them. It is true that you don’t need to learn C# to write code for .NET. Microsoft has extended C++ and made substantial changes to Visual Basic to turn it into a more powerful language to enable code written in either of these languages to target the .NET environment. These other languages, however, are hampered by the legacy of having evolved over the years rather than having been written from the start with today’s technology in mind. This book can equip you to program in C#, while at the same time provides the necessary background in how the .NET architecture works. You not only cover the fundamentals of the C# language, but also see examples of applications that use a variety of related technologies, including database access, dynamic web pages, advanced graphics, and directory access. While the Windows API evolved and was extended since the early days of Windows NT in 1993, the .NET Framework offered a major change on how programs are written since the year 2002; now, starting with the year 2012, we have the days of the next big change. Do such changes happen every 10 years? Windows 8 offers a new API: the Windows Runtime (WinRT) for Windows Store apps. This runtime is a native API (like the Windows API) that is not build with the .NET runtime as its core, but offers great new features that are based on ideas of .NET. Windows 8 includes the first release of this API available for modern-style apps. Although this is not based on .NET, you still can use a subset of .NET with Windows Store apps, and write the apps with C#. This new runtime evolves, and with Windows 8.1 version 2 is included. This book will give you a start in writing Windows Store apps with C# and WinRT. Advantages of .NET So far, you’ve read in general terms about how great .NET is, but it can help to make your life as a developer easier. This section briefly identifies some of the features of .NET: ➤➤ Object-oriented programming — Both the .NET Framework and C# are entirely based on object- oriented principles from the start. ➤➤ Good design — A base class library, which is designed from the ground up in a highly intuitive way. flast.indd 24 30-01-2014 20:59:29 xxv introduction ➤➤ Language independence — With .NET, all the languages — Visual Basic, C#, and managed C++ — compile to a common Intermediate Language. This means that languages are interoperable in a way that has not been seen before. ➤➤ Better support for dynamic web pages — Though Classic ASP offered a lot of flexibility, it was also inefficient because of its use of interpreted scripting languages, and the lack of object-oriented design often resulted in messy ASP code. .NET offers an integrated support for web pages, using ASP .NET. With ASP.NET, code in your pages is compiled and may be written in a .NET-aware high-level language such as C# or Visual Basic 2013. .NET now takes it even further with outstanding support for the latest web technologies such as Ajax and jQuery. ➤➤ Efficient data access — A set of .NET components, collectively known as ADO.NET, provides efficient access to relational databases and a variety of data sources. Components are also available to enable access to the file system and to directories. In particular, XML support is built into .NET, enabling you to manipulate data, which may be imported from or exported to non-Windows platforms. ➤➤ Code sharing — .NET has completely revamped the way that code is shared between applications, introducing the concept of the assembly, which replaces the traditional DLL. Assemblies have formal facilities for versioning, and different versions of assemblies can exist side by side. ➤➤ Improved security — Each assembly can also contain built-in security information that can indicate precisely who or what category of user or process is allowed to call which methods on which classes. This gives you a fine degree of control over how the assemblies that you deploy can be used. ➤➤ Zero-impact installation — There are two types of assemblies: shared and private. Shared assemblies are common libraries available to all software, whereas private assemblies are intended only for use with particular software. A private assembly is entirely self-contained, so the process to install it is simple. There are no registry entries; the appropriate files are simply placed in the appropriate folder in the file system. ➤➤ Support for web services — .NET has fully integrated support for developing web services as easily as you would develop any other type of application. ➤➤ Visual Studio 2013 — .NET comes with a developer environment, Visual Studio 2013, which can cope equally well with C++, C#, and Visual Basic 2013, as well as with ASP.NET or XML code. Visual Studio 2013 integrates all the best features of the respective language-specific environments of all the previous versions of this amazing IDE. ➤➤ C# — C# is a powerful and popular object-oriented language intended for use with .NET. You look more closely at the benefits of the .NET architecture in Chapter 1, “.NET Architecture.” What’s New in the .NET Framework 4.5 and .NET 4.5.1 The first version of the .NET Framework (1.0) was released in 2002 to much enthusiasm. The .NET Framework 2.0 was introduced in 2005 and was considered a major release of the Framework. The major new feature of 2.0 was generics support in C# and the runtime (IL code changed for generics), and new classes and interfaces. .NET 3.0 was based on the 2.0 runtime and introduced a new way to create UIs (WPF with XAML and vector-based graphics instead of pixel-based), and a new communication technology (WCF). .NET 3.5 together with C# 3.0 introduced LINQ, one query syntax that can be used for all data sources. .NET 4.0 was another major release of the product that also brought a new version of the runtime (4.0) and a new version of C# (4.0) to offer dynamic language integration and a huge new library for parallel programming. The .NET Framework 4.5 is based on an updated version of the 4.0 runtime with many outstanding new features. The .NET Framework 4.5.1 gives some small increments. However, with more flast.indd 25 30-01-2014 20:59:29 xxvi introduction and more libraries that are part of the .NET Framework being distributed as NuGet packages, more and more features are delivered out of band to the .NET Framework. For example, the Entity Framework, ASP .NET Web API, and other .NET libraries got huge improvements. With each release of the Framework, Microsoft has always tried to ensure that there were minimal breaking changes to code developed. Thus far, Microsoft has been successful at this goal. The following section details some of the changes that are new to C# 5.0 and the .NET Framework 4.5.1. Asynchronous Programming Blocking the UI is unfriendly to the user; the user becomes impatient if the UI does not react. Maybe you’ve had this experience with Visual Studio as well. Good news: Visual Studio has become a lot better in reacting faster in many scenarios. The .NET Framework always offered calling methods asynchronously. However, using synchronous methods was a lot easier than calling their asynchronous variant. This changed with C# 5.0. Programming asynchronously has become as easy as writing synchronous programs. New C# keywords are based on the .NET Parallel Library that is available since .NET 4.0. Now the language offers productivity features. Windows Store Apps and the Windows Runtime Windows Store apps can be programmed with C# using the Windows Runtime and a subset of the .NET Framework. The Windows Runtime is a new native API that offers classes, methods, properties, and events that look like .NET; although it is native. For using language projection features, the .NET runtime has been enhanced. With .NET 4.5, the .NET 4.0 runtime gets an in-place update. Enhancements with Data Access The ADO.NET Entity Framework offered important new features. Its version changed from 4.0 with .NET 4.0 to 5.0 with .NET 4.5, and to 6.0 with .NET 4.5.1. After the release of .NET 4.0, the Entity Framework already received updates with versions 4.1, 4.2, and 4.3. New features such as Code First, spatial types, using enums, and table-valued functions are now available. Enhancements with WPF For programming Windows desktop applications, WPF has been enhanced. Now you can fill collections from a non-UI thread; the ribbon control is now part of the framework; weak references with events have been made easier; validation can be done asynchronously with the INotifyDataErrorInfo interface; and live shaping allows easy dynamic sorting and grouping with data that changes. a sP.ne t m vc Visual Studio 2010 included ASP.NET MVC 2.0. With the release of Visual Studio 2013, ASP.NET MVC 5.0 is available. ASP.NET MVC supplies you with the means to create ASP.NET using the model-view-controller model that many developers expect. ASP.NET MVC provides developers with testability, flexibility, and maintainability in the applications they build. ASP.NET MVC is not meant to be a replacement for ASP.NET Web Forms but is simply a different way to construct your applications. flast.indd 26 30-01-2014 20:59:29 xxvii introduction Where C# Fits In In one sense, C# is the same thing to programming languages that .NET is to the Windows environment. Just as Microsoft has been adding more and more features to Windows and the Windows API over the past 15 years, Visual Basic 2013 and C++ have undergone expansion. Although Visual Basic and C++ have resulted in hugely powerful languages, both languages also suffer from problems because of the legacies left over from the way they evolved. For Visual Basic 6 and earlier versions, the main strength of the language was that it was simple to understand and made many programming tasks easy, largely hiding the details of the Windows API and the COM component infrastructure from the developer. The downside to this was that Visual Basic was never truly object-oriented, so large applications quickly became disorganized and hard to maintain. Also, because Visual Basic’s syntax was inherited from early versions of BASIC (which, in turn, was designed to be intuitively simple for beginning programmers to understand, rather than to write large commercial applications), it didn’t lend itself to well-structured or object-oriented programs. C++, on the other hand, has its roots in the ANSI C++ language definition. It is not completely ANSI- compliant for the simple reason that Microsoft first wrote its C++ compiler before the ANSI definition had become official, but it comes close. Unfortunately, this has led to two problems. First, ANSI C++ has its roots in a decade-old state of technology, and this shows up in a lack of support for modern concepts (such as Unicode strings and generating XML documentation) and for some archaic syntax structures designed for the compilers of yesteryear (such as the separation of declaration from definition of member functions). Second, Microsoft has been simultaneously trying to evolve C++ into a language designed for high- performance tasks on Windows, and to achieve that, it has been forced to add a huge number of Microsoft- specific keywords as well as various libraries to the language. The result is that on Windows, the language has become a complete mess. Just ask C++ developers how many definitions for a string they can think of: char*, LPTSTR, string, CString (MFC version), CString (WTL version), wchar_t*, OLECHAR*, and so on. Now enters .NET — a completely revolutionary environment that has brought forth new extensions to both languages. Microsoft has gotten around this by adding yet more Microsoft-specific keywords to C++ and by completely revamping Visual Basic to the current Visual Basic 2013, a language that retains some of the basic VB syntax but that is so different in design from the original VB that it can be considered, for all practical purposes, a new language. It is in this context that Microsoft has provided developers an alternative — a language designed specifically for .NET and designed with a clean slate. C# is the result. Officially, Microsoft describes C# as a “simple, modern, object-oriented, and type-safe programming language derived from C and C++.” Most independent observers would probably change that to “derived from C, C++, and Java.” Such descriptions are technically accurate but do little to convey the beauty or elegance of the language. Syntactically, C# is similar to both C++ and Java, to such an extent that many keywords are the same, and C# also shares the same block structure with braces ({}) to mark blocks of code and semicolons to separate statements. The first impression of a piece of C# code is that it looks quite like C++ or Java code. Beyond that initial similarity, however, C# is a lot easier to learn than C++ and of comparable difficulty to Java. Its design is more in tune with modern developer tools than both of those other languages, and it has been designed to provide, simultaneously, the ease of use of Visual Basic and the high-performance, low-level memory access of C++, if required. Some of the features of C# follow: ➤➤ Full support for classes and object-oriented programming, including interface and implementation inheritance, virtual functions, and operator overloading. ➤➤ A consistent and well-defined set of basic types. ➤➤ Built-in support for an automatic generation of XML documentation. ➤➤ Automatic cleanup of dynamically allocated memory. flast.indd 27 30-01-2014 20:59:29 xxviii introduction ➤➤ The facility to mark classes or methods with user-defined attributes. This can be useful for documentation and can have some effects on compilation (for example, marking methods to be compiled only in debug builds). ➤➤ Full access to the .NET base class library and easy access to the Windows API (if you need it, which will not be often). ➤➤ Pointers and direct memory access are available if required, but the language has been designed in such a way that you can work without them in almost all cases. ➤➤ Support for properties and events in the style of Visual Basic. ➤➤ Just by changing the compiler options, you can compile either to an executable or to a library of .NET components that can be called up by other code in the same way as ActiveX controls (COM components). ➤➤ C# can be used to write ASP.NET dynamic web pages and XML web services. Most of these statements, it should be pointed out, also apply to Visual Basic 2013 and Managed C++. Because C# is designed from the start to work with .NET, however, means that its support for the features of .NET is both more complete and offered within the context of a more suitable syntax than those of other languages. Although the C# language is similar to Java, there are some improvements; in particular, Java is not designed to work with the .NET environment. Before leaving the subject, you must understand a couple of limitations of C#. The one area the language is not designed for is time-critical or extremely high-performance code — the kind where you are worried about whether a loop takes 1,000 or 1,050 machine cycles to run through, and you need to clean up your resources the millisecond they are no longer needed. C++ is likely to continue to reign supreme among low-level languages in this area. C# lacks certain key facilities needed for extremely high-performance apps, including the capability to specify inline functions and destructors guaranteed to run at particular points in the code. However, the proportions of applications that fall into this category are low. What You Need to Write and Run C# Code The .NET Framework 4.5.1 can run on the client operating systems Windows Vista, 7, 8, 8.1, and the server operating systems Windows Server 2008, 2008 R2, 2012, and 2012 R2. To write code using .NET, you need to install the .NET 4.5.1 SDK. In addition, unless you intend to write your C# code using a text editor or some other third-party developer environment, you almost certainly also want Visual Studio 2013. The full SDK is not needed to run managed code, but the .NET runtime is needed. You may find you need to distribute the .NET runtime with your code for the benefit of those clients who do not have it already installed. What This Book Covers This book starts by reviewing the overall architecture of .NET in Chapter 1 to give you the background you need to write managed code. After that, the book is divided into a number of sections that cover both the C# language and its application in a variety of areas. Part I: The C# Language This section gives a good grounding in the C# language. This section doesn’t presume knowledge of any particular language; although, it does assume you are an experienced programmer. You start by looking at flast.indd 28 30-01-2014 20:59:29 xxix introduction C#’s basic syntax and data types and then explore the object-oriented features of C# before looking at more advanced C# programming topics. Part II: Visual Studio This section looks at the main IDE utilized by C# developers worldwide: Visual Studio 2013. The two chapters in this section look at the best way to use the tool to build applications based on the .NET Framework 4.5.1. In addition, this section also focuses on the deployment of your projects. Part III: Foundation In this section, you look at the principles of programming in the .NET environment. In particular, you look at security, threading, localization, transactions, how to build Windows services, and how to generate your own libraries as assemblies, among other topics. One part is interaction with native code and assemblies using Platform Invoke and COM interop. This section also gives information about how the Windows Runtime differs from .NET and how to start writing Windows 8–style programs. Part IV: Data Here, you look at accessing data using ADO.NET and learn about the ADO.NET Entity Framework. You can use core ADO.NET to get the best performance; the ADO.NET Entity Framework offers ease of use with mapping objects to relations. Now, different programming models with Model First, Database First, and Code First are available that are all discussed. This part also extensively covers support in .NET for XML, using LINQ to query XML data sources. Part V: Presentation This section starts by showing you how to build applications based upon the Windows Presentation Foundation. Not only are different control types, styles, resources, and data binding covered, but you can also read about creating fixed and flow documents, and printing. Here, you can also read about creating Windows Store apps, use of pictures for a nicer UI, grids, and contracts to interact with other applications. Finally, this section includes coverage of the tremendous number of features that ASP.NET offers, building websites with ASP.NET Web Forms, ASP.NET MVC, and dynamic data. Part VI: Communication This section is all about communication. It covers services for platform-independent communication using Windows Communication Foundation (WCF) and the ASP.NET Web API. With Message Queuing, asynchronous disconnected communication is shown. This section looks at utilizing the Windows Workflow Foundation and peer-to-peer networking. Conventions To help you get the most from the text and keep track of what’s happening, a number of conventions are used throughout the book. Warning  Warnings hold important, not-to-be-forgotten information that is directly relevant to the surrounding text. flast.indd 29 30-01-2014 20:59:29 xxx introduction Note  Notes indicate notes, tips, hints, tricks, and/or asides to the current discussion. As for styles in the text: ➤➤ We highlight new terms and important words when we introduce them. ➤➤ We show keyboard strokes like this: Ctrl+A. ➤➤ We show filenames, URLs, and code within the text like so: persistence.properties. ➤➤ We present code in two different ways: We use a monofont type with no highlighting for most code examples. We use bold to emphasize code that's particularly important in the present context or to show changes from a previous code snippet. Source Code As you work through the examples in this book, you may choose either to type in all the code manually or to use the source code files that accompany the book. All the source code used in this book is available for download at www.wrox.com/go/procsharp. When at the site, simply locate the book’s title (either by using the Search box or by using one of the title lists) and click the Download Code link on the book’s detail page to obtain all the source code for the book. Note  Because many books have similar titles, you may find it easiest to search by ISBN; this book’s ISBN is 978-1-118-83303-2. After you download the code, just decompress it with your favorite compression tool. Alternatively, you can go to the main Wrox code download page at http://www.wrox.com/dynamic/books/download.aspx to see the code available for this book and all other Wrox books. Errata We make every effort to ensure that there are no errors in the text or in the code. However, no one is perfect, and mistakes do occur. If you find an error in one of our books, like a spelling mistake or faulty piece of code, we would be grateful for your feedback. By sending in errata you may save another reader hours of frustration, and at the same time you can help provide even higher quality information. To find the errata page for this book, go to http://www.wrox.com and locate the title using the Search box or one of the title lists. Then, on the book details page, click the Book Errata link. On this page you can view all errata that has been submitted for this book and posted by Wrox editors. A complete book list including links to each book’s errata is also available at www.wrox.com/misc-pages/booklist.shtml. If you don’t spot “your” error on the Book Errata page, go to www.wrox.com/contact/techsupport .shtml and complete the form there to send us the error you have found. We’ll check the information and, if appropriate, post a message to the book’s errata page and fix the problem in subsequent editions of the book. flast.indd 30 30-01-2014 20:59:30 xxxi introduction p2p.wrox.com For author and peer discussion, join the P2P forums at p2p.wrox.com. The forums are a web-based system for you to post messages relating to Wrox books and related technologies and interact with other readers and technology users. The forums offer a subscription feature to e-mail you topics of interest of your choosing when new posts are made to the forums. Wrox authors, editors, other industry experts, and your fellow readers are present on these forums. At http://p2p.wrox.com you can find a number of different forums to help you not only as you read this book, but also as you develop your own applications. To join the forums, just follow these steps: 1. Go to p2p.wrox.com and click the Register link. 2. Read the terms of use and click Agree. 3. Complete the required information to join and any optional information you want to provide, and click Submit. 4. You will receive an e-mail with information describing how to verify your account and complete the joining process. Note  You can read messages in the forums without joining P2P but to post your own messages, you must join. After you join, you can post new messages and respond to messages other users post. You can read messages at any time on the web. If you want to have new messages from a particular forum e-mailed to you, click the Subscribe to this Forum icon by the forum name in the forum listing. For more information about how to use the Wrox P2P, read the P2P FAQs for answers to questions about how the forum software works as well as many common questions specific to P2P and Wrox books. To read the FAQs, click the FAQ link on any P2P page. flast.indd 31 30-01-2014 20:59:30 flast.indd 32 30-01-2014 20:59:30 PART I The C# Language ➤ CHAPTER 1: .NET Architecture ➤ CHAPTER 2: Core C# ➤ CHAPTER 3: Objects and Types ➤ CHAPTER 4: Inheritance ➤ CHAPTER 5: Generics ➤ CHAPTER 6: Arrays and Tuples ➤ CHAPTER 7: Operators and Casts ➤ CHAPTER 8: Delegates, Lambdas, and Events ➤ CHAPTER 9: Strings and Regular Expressions ➤ CHAPTER 10: Collections ➤ CHAPTER 11: Language Integrated Query ➤ CHAPTER 12: Dynamic Language Extensions ➤ CHAPTER 13: Asynchronous Programming ➤ CHAPTER 14: Memory Management and Pointers ➤ CHAPTER 15: Refl ection ➤ CHAPTER 16: Errors and Exceptions c01.indd 1 30-01-2014 19:56:47 c01.indd 2 30-01-2014 19:56:47 .NET Architecture wHAT’s in THis CHAPTER? ➤➤ Compiling and running code that targets .NET ➤➤ Advantages of Microsoft Intermediate Language (MSIL) ➤➤ Value and reference types ➤➤ Data typing ➤➤ Understanding error handling and attributes ➤➤ Assemblies, .NET base classes, and namespaces CodE downLoAds FoR THis CHAPTER There are no code downloads for this chapter. THE RELATionsHiP oF C# To .nET This book emphasizes that the C# language must be considered in parallel with the .NET Framework, rather than viewed in isolation. The C# compiler specifi cally targets .NET, which means that all code written in C# always runs using the .NET Framework. This has two important consequences for the C# language: 1. The architecture and methodologies of C# refl ect the underlying methodologies of .NET. 2. In many cases, specifi c language features of C# actually depend on features of .NET or of the .NET base classes. Because of this dependence, you must gain some understanding of the architecture and methodology of .NET before you begin C# programming, which is the purpose of this chapter. C# is a programming language newly designed for .NET and is signifi cant in two respects: ➤➤ It is specifi cally designed and targeted for use with Microsoft’s .NET Framework (a feature-rich platform for the development, deployment, and execution of distributed applications). ➤➤ It is a language based on the modern object-oriented design methodology, and when designing it Microsoft learned from the experience of all the other similar languages that have been around since object-oriented principles came to prominence 20 years ago. 1 c01.indd 3 30-01-2014 19:56:50 4  ❘  CHAPTER 1  .NET Architecture C# is a language in its own right. Although it is designed to generate code that targets the .NET environment, it is not part of .NET. Some features are supported by .NET but not by C#, and you might be surprised to learn that some features of the C# language are not supported by .NET or by MSIL (for example, some instances of operator overloading). However, because the C# language is intended for use with .NET, you must understand this Framework if you want to develop applications in C# effectively. Therefore, this chapter takes some time to peek underneath the surface of .NET. The Common Language Runtime Central to the .NET Framework is its runtime execution environment, known as the Common Language Runtime (CLR) or the .NET runtime. Code running under the control of the CLR is often termed managed code. However, before it can be executed by the CLR, any source code that you develop (in C# or some other language) needs to be compiled. Compilation occurs in two steps in .NET: 1. Compilation of source code to Microsoft Intermediate Language (IL). 2. Compilation of IL to platform-specific code by the CLR. This two-stage compilation process is important because the existence of the Microsoft Intermediate Language is the key to providing many of the benefits of .NET. IL shares with Java byte code the idea that it is a low-level language with a simple syntax (based on numeric codes rather than text), which can be quickly translated into native machine code. Having this well-defined universal syntax for code has significant advantages: platform independence, performance improvement, and language interoperability. Platform Independence First, platform independence means that the same file containing byte code instructions can be placed on any platform; at runtime, the final stage of compilation can then be easily accomplished so that the code can run on that particular platform. In other words, by compiling to IL you obtain platform independence for .NET in much the same way as compiling to Java byte code gives Java platform independence. The platform independence of .NET is only theoretical at present because a complete implementation of .NET is available only for Windows. However, a partial, cross-platform implementation is available (see, for example, the Mono project, an effort to create an open source implementation of .NET, at www.go-mono.com). You can also use C# on iPhone and Android devices by using tools and libraries from Xamarin (www.xamarin.com). Performance Improvement Although previously compared to Java, IL is actually a bit more ambitious than Java byte code. IL is always Just-in-Time compiled (known as JIT compilation), whereas Java byte code was often interpreted. One of the disadvantages of Java was that, on execution, the process to translate from Java byte code to native executable resulted in a loss of performance (with the exception of more recent cases in which Java is JIT compiled on certain platforms). Instead of compiling the entire application at one time (which could lead to a slow startup time), the JIT compiler simply compiles each portion of code as it is called (just in time). When code has been compiled once, the resultant native executable is stored until the application exits so that it does not need to be recompiled the next time that portion of code is run. Microsoft argues that this process is more efficient than compiling the entire application code at the start because of the likelihood that large portions of any application code will not actually be executed in any given run. Using the JIT compiler, such code can never be compiled. c01.indd 4 30-01-2014 19:56:50 The Common Language Runtime  ❘  5 This explains why you can expect that execution of managed IL code will be almost as fast as executing native machine code. What it does not explain is why Microsoft expects that you get a performance improvement. The reason given for this is that because the final stage of compilation takes place at runtime, the JIT compiler knows exactly what processor type the program runs on. This means that it can optimize the final executable code to take advantage of any features or particular machine code instructions offered by that particular processor. Traditional compilers optimize the code, but they can perform optimizations that are only independent of the particular processor that the code runs on. This is because traditional compilers compile to native executable code before the software is shipped. This means that the compiler does not know what type of processor the code runs on beyond basic generalities, such as that it is an x86-compatible processor or an Alpha processor. Language Interoperability The use of IL not only enables platform independence, but it also facilitates language interoperability. Simply put, you can compile to IL from one language, and this compiled code should then be interoperable with code that has been compiled to IL from another language. You are probably now wondering which languages aside from C# are interoperable with .NET. The following sections briefly discuss how some of the other common languages fit into .NET. Visual Basic 2013 Visual Basic .NET 2002 underwent a complete revamp from Visual Basic 6 to bring it up to date with the first version of the .NET Framework. The Visual Basic language had dramatically evolved from VB6, which meant that VB6 was not a suitable language to run .NET programs. For example, VB6 is heavily integrated into Component Object Model (COM) and works by exposing only event handlers as source code to the developer — most of the background code is not available as source code. Not only that, it does not support implementation inheritance, and the standard data types that Visual Basic 6 uses are incompatible with .NET. Visual Basic 6 was upgraded to Visual Basic .NET in 2002, and the changes that were made to the language are so extensive you might as well regard Visual Basic as a new language. Since then, Visual Basic evolved with many language enhancements — much as C# did. Visual Basic and C# are very similar in their features because they are developed from the same product team within Microsoft. As time has passed, Visual Basic acquired features that were once only available with C#, and C# acquired features that were once only available with Visual Basic. Nowadays there are only minor differences between Visual Basic and C#, and mainly it’s a matter of taste to use curly brackets or END statements. Visual C++ 2013 Visual C++ 6 already had a large number of Microsoft-specific extensions on Windows. With Visual C++ .NET, extensions have been added to support the .NET Framework. This means that existing C++ source code will continue to compile to native executable code without modification. It also means, however, that it will run independently of the .NET runtime. If you want your C++ code to run within the .NET Framework, you can simply add the following line to the beginning of your code: #using You can also pass the flag /clr to the compiler, which then assumes that you want to compile to managed code and will hence emit IL instead of native machine code. The interesting thing about C++ is that when you compile to managed code, the compiler can emit IL that contains an embedded native executable. This means that you can mix managed types and unmanaged types in your C++ code. Thus, the managed C++ code, class MyClass { c01.indd 5 30-01-2014 19:56:50 6  ❘  CHAPTER 1  .NET Architecture defines a plain C++ class, whereas the code, ref class MyClass { gives you a managed class, just as if you had written the class in C# or Visual Basic 2013. The advantage to use managed C++ over C# code is that you can call unmanaged C++ classes from managed C++ code without resorting to COM interop. The compiler raises an error if you attempt to use features not supported by .NET on managed types (for example, templates or multiple inheritances of classes). You can also find that you need to use nonstandard C++ features when using managed classes. Writing C++ programs that use .NET gives you different variants of interop scenarios. With the compiler setting /clr for Common Language Runtime Support, you can completely mix all native and managed C++ features. Other options such as /clr:safe and /clr:pure restrict the use of native C++ pointers and thus enable writing safe code like with C# and Visual Basic. Visual C++ 2013 enables you to create programs for the Windows Runtime (WinRT) with Windows 8.1. This way C++ does not use managed code but instead accesses the WinRT natively. Visual F# F# is a strongly typed functional programming language. This language has strong support inside Visual Studio. Being a functional programming language it looks very different from C#. For example, declaring a type Person with the members FirstName and LastName looks like this: module PersonSample type Person(firstName : string, lastName : string) = member this.FirstName = firstName member this.LastName = lastName And using the Person type makes use of the let keyword. printfn writes results to the console: open PersonSample [] let main argv = let p = Person("Sebastian", "Vettel") let first = p.firstName let last = p.lastName printfn "%s %s" first last 0 // return an integer exit code F# can use all the types you create with C#, and vice versa. The advantage of F# is that it is a functional language instead of object-oriented, and this helps with programming of complex algorithms — for example, for financial and scientific applications. COM and COM+ Technically speaking, COM and COM+ are not technologies targeted at .NET  — components based on them cannot be compiled into IL. (Although you can do so to some degree using managed C++ if the original COM component were written in C++). However, COM+ remains an important tool because its features are not duplicated in .NET. Also, COM components can still work — and .NET incorporates COM interoperability features that make it possible for managed code to call up COM components and vice versa (discussed in Chapter 23, “Interop”). In general, you will probably find it more convenient for most purposes to code new components as .NET components so that you can take advantage of the .NET base classes and the other benefits of running as managed code. c01.indd 6 30-01-2014 19:56:50 A Closer Look at Intermediate Language  ❘  7 Windows Runtime Windows 8 offers a new runtime used by Windows Store apps. Some parts of this runtime can be used with desktop applications as well. You can use this runtime from Visual Basic, C#, C++, and JavaScript. When using the runtime with these different environments, it looks different. Using it from C# it looks like classes from the .NET Framework. Using it from JavaScript it looks like what JavaScript developers are used to with JavaScript libraries. And using it from C++, methods looks like the Standard C++ Library. This is done by using language projection. The Windows Runtime and how it looks from C# is discussed in Chapter 31, “Windows Runtime.” A Closer Look at Intermediate Language From what you learned in the previous section, Microsoft Intermediate Language obviously plays a fundamental role in the .NET Framework. It makes sense now to take a closer look at the main features of IL because any language that targets .NET logically needs to support these characteristics. Here are the important features of IL: ➤➤ Object orientation and the use of interfaces ➤➤ Strong distinction between value and reference types ➤➤ Strong data typing ➤➤ Error handling using exceptions ➤➤ Use of attributes The following sections explore each of these features. Support for Object Orientation and Interfaces The language independence of .NET does have some practical limitations. IL is inevitably going to implement some particular programming methodology, which means that languages targeting it need to be compatible with that methodology. The particular route that Microsoft has chosen to follow for IL is that of classic object-oriented programming, with single implementation inheritance of classes. In addition to classic object-oriented programming, IL also brings in the idea of interfaces, which saw their first implementation under Windows with COM. Interfaces built using .NET produce interfaces that are not the same as COM interfaces. They do not need to support any of the COM infrastructure. (For example, they are not derived from IUnknown and do not have associated globally unique identifiers, more commonly known as GUIDs.) However, they do share with COM interfaces the idea that they provide a contract, and classes that implement a given interface must provide implementations of the methods and properties specified by that interface. You have now seen that working with .NET means compiling to IL, and that in turn means that you need to understand traditional object-oriented methodologies. However, that alone is not sufficient to give you language interoperability. After all, C++ and Java both use the same object-oriented paradigms but are still not regarded as interoperable. You need to look a little more closely at the concept of language interoperability. So what exactly is language interoperability? After all, COM enabled components written in different languages to work together in the sense of calling each other’s methods. What was inadequate about that? COM, by virtue of being a binary standard, did enable components to instantiate other components and call methods or properties against them, without worrying about the language in which the respective components were written. To achieve this, however, each object had to be instantiated through the COM runtime and accessed through an interface. Depending on the threading models of the relative components, there may have been large performance losses associated with marshaling data between apartments or running components or both on different threads. In the extreme case of components hosted as an executable rather than DLL files, separate processes would c01.indd 7 30-01-2014 19:56:51 8  ❘  CHAPTER 1  .NET Architecture need to be created to run them. The emphasis was very much that components could talk to each other but only via the COM runtime. In no way with COM did components written in different languages directly communicate with each other, or instantiate instances of each other — it was always done with COM as an intermediary. Not only that, but the COM architecture did not permit implementation inheritance, which meant that it lost many of the advantages of object-oriented programming. An associated problem was that, when debugging, you would still need to debug components written in different languages independently. It was not possible to step between languages in the debugger. Therefore, what you actually mean by language interoperability is that classes written in one language should talk directly to classes written in another language. In particular: ➤➤ A class written in one language can inherit from a class written in another language. ➤➤ The class can contain an instance of another class, no matter what the languages of the two classes are. ➤➤ An object can directly call methods against another object written in another language. ➤➤ Objects (or references to objects) can be passed around between methods. ➤➤ When calling methods between languages, you can step between the method calls in the debugger, even when this means stepping between source code written in different languages. This is all quite an ambitious aim, but amazingly .NET and IL have achieved it. In the case of stepping between methods in the debugger, this facility is actually offered by the Visual Studio integrated development environment (IDE) rather than by the CLR. Distinct Value and Reference Types As with any programming language, IL provides a number of predefined primitive data types. One characteristic of IL, however, is that it makes a strong distinction between value and reference types. Value types are those for which a variable directly stores its data, whereas reference types are those for which a variable simply stores the address at which the corresponding data can be found. In C++ terms, using reference types is similar to accessing a variable through a pointer, whereas for Visual Basic the best analogy for reference types are objects, which in Visual Basic 6 are always accessed through references. IL also lays down specifications about data storage: Instances of reference types are always stored in an area of memory known as the managed heap, whereas value types are normally stored on the stack. (Although if value types are declared as fields within reference types, they will be stored inline on the heap.) Chapter 2, “Core C#,” discusses the stack and the managed heap and how they work. Strong Data Typing One important aspect of IL is that it is based on exceptionally strong data typing. That means that all variables are clearly marked as being of a particular, specific data type. (There is no room in IL, for example, for the Variant data type recognized by Visual Basic and scripting languages.) In particular, IL does not normally permit any operations that result in ambiguous data types. For instance, Visual Basic 6 developers are used to passing variables around without worrying too much about their types because Visual Basic 6 automatically performs type conversion. C++ developers are used to routinely casting pointers between different types. Performing this kind of operation can be great for performance, but it breaks type safety. Hence, it is permitted only under certain circumstances in some of the languages that compile to managed code. Indeed, pointers (as opposed to references) are permitted only in marked blocks of code in C#, and not at all in Visual Basic. (Although they are allowed in managed C++.) Using pointers in your code causes it to fail the memory type-safety checks performed by the CLR. Some languages compatible with .NET, such as Visual Basic 2010, still allow some laxity in typing but only because the compilers behind the scenes ensure that the type safety is enforced in the emitted IL. Although enforcing type safety might initially appear to hurt performance, in many cases the benefits gained from the services provided by .NET that rely on type safety far outweigh this performance loss. Such services include the following: c01.indd 8 30-01-2014 19:56:51 A Closer Look at Intermediate Language  ❘  9 ➤➤ Language interoperability ➤➤ Garbage collection ➤➤ Security ➤➤ Application domains The following sections take a closer look at why strong data typing is particularly important for these features of .NET. Strong Data Typing as a Key to Language Interoperability If a class is to derive from or contains instances of other classes, it needs to know about all the data types used by the other classes. This is why strong data typing is so important. Indeed, it is the absence of any agreed-on system for specifying this information in the past that has always been the real barrier to inheritance and interoperability across languages. This kind of information is simply not present in a standard executable file or DLL. Suppose that one of the methods of a Visual Basic 2013 class is defined to return an Integer — one of the standard data types available in Visual Basic 2013. C# simply does not have any data type of that name. Clearly, you can derive from the class, use this method, and use the return type from C# code only if the compiler knows how to map Visual Basic 2013’s Integer type to some known type defined in C#. So, how is this problem circumvented in .NET? Common Type System This data type problem is solved in .NET using the Common Type System (CTS). The CTS defines the predefined data types available in IL so that all languages that target the .NET Framework can produce compiled code ultimately based on these types. For the previous example, Visual Basic 2013’s Integer is actually a 32-bit signed integer, which maps exactly to the IL type known as Int32. Therefore, this is the data type specified in the IL code. Because the C# compiler is aware of this type, there is no problem. At source code-level, C# refers to Int32 with the keyword int, so the compiler simply treats the Visual Basic 2013 method as if it returned an int. The CTS does not specify merely primitive data types but a rich hierarchy of types, which includes well- defined points in the hierarchy at which code is permitted to define its own types. The hierarchical structure of the CTS reflects the single-inheritance object-oriented methodology of IL, and resembles Figure 1-1. Built-in Value Types User-defined Value Types Value Type Pointer Types Type Reference Type Enumerations Interface Types Self-describing Types ArraysClass Types User-defined Reference Types Delegates Boxed Value Types FIGURE 1-1 c01.indd 9 30-01-2014 19:56:53 10  ❘  CHAPTER 1  .NET Architecture All of the built-in value types aren’t here because they are covered in detail in Chapter 3, “Objects and Types.” In C#, each predefined type is recognized by the compiler maps onto one of the IL built-in types. The same is true in Visual Basic 2013. Common Language Specification The Common Language Specification (CLS) works with the CTS to ensure language interoperability. The CLS is a set of minimum standards that all compilers targeting .NET must support. Because IL is a rich language, writers of most compilers prefer to restrict the capabilities of a given compiler to support only a subset of the facilities offered by IL and the CTS. That is fine as long as the compiler supports everything defined in the CLS. For example, take case sensitivity. IL is case-sensitive. Developers who work with case-sensitive languages regularly take advantage of the flexibility that this case sensitivity gives them when selecting variable names. Visual Basic 2013, however, is not case-sensitive. The CLS works around this by indicating that CLS- compliant code should not expose any two names that differ only in their case. Therefore, Visual Basic 2013 code can work with CLS-compliant code. This example shows that the CLS works in two ways: 1. Individual compilers do not need to be powerful enough to support the full features of .NET — this should encourage the development of compilers for other programming languages that target .NET. 2. If you restrict your classes to exposing only CLS-compliant features, then it guarantees that code written in any other compliant language can use your classes. The beauty of this idea is that the restriction to using CLS-compliant features applies only to public and protected members of classes and public classes. Within the private implementations of your classes, you can write whatever non-CLS code you want because code in other assemblies (units of managed code; see later in the section Assemblies) cannot access this part of your code. Without going into the details of the CLS specifications here, in general, the CLS does not affect your C# code much because of the few non-CLS-compliant features of C#. NOTE  It is perfectly acceptable to write non-CLS-compliant code. However, if you do, the compiled IL code is not guaranteed to be fully language interoperable. You can mark your assembly or types to be CLS-compliant by applying the CLSCompliant attri- bute. Using this attribute, the compiler checks for compliance. If a non-compliant data type is used with the signature of a public method (for example, uint), you will get a compiler warning that the method is not CLS compliant. Garbage Collection The garbage collector is .NET’s answer to memory management and in particular to the question of what to do about reclaiming memory that running applications ask for. Up until now, two techniques have been used on the Windows platform for de-allocating memory that processes have dynamically requested from the system: ➤➤ Make the application code do it all manually. ➤➤ Make objects maintain reference counts. Having the application code responsible for de-allocating memory is the technique used by lower-level, high- performance languages such as C++. It is efficient and has the advantage that (in general) resources are never occupied for longer than necessary. The big disadvantage, however, is the frequency of bugs. Code that requests memory also should explicitly inform the system when it no longer requires that memory. However, it is easy to overlook this, resulting in memory leaks. c01.indd 10 30-01-2014 19:56:53 A Closer Look at Intermediate Language  ❘  11 Although modern developer environments do provide tools to assist in detecting memory leaks, they remain difficult bugs to track down. That’s because they have no effect until so much memory has been leaked that Windows refuses to grant any more to the process. By this point, the entire computer may have appreciably slowed down due to the memory demands made on it. Maintaining reference counts is favored in COM. The idea is that each COM component maintains a count of how many clients are currently maintaining references to it. When this count falls to zero, the component can destroy itself and free up associated memory and resources. The problem with this is that it still relies on the good behavior of clients to notify the component that they have finished with it. It takes only one client not to do so, and the object sits in memory. In some ways, this is a potentially more serious problem than a simple C++-style memory leak because the COM object may exist in its own process, which means that it can never be removed by the system. (At least with C++ memory leaks, the system can reclaim all memory when the process terminates.) The .NET runtime relies on the garbage collector instead. The purpose of this program is to clean up memory. The idea is that all dynamically requested memory is allocated on the heap. (That is true for all languages; although in the case of .NET, the CLR maintains its own managed heap for .NET applications to use.) Sometimes, when .NET detects that the managed heap for a given process is becoming full and therefore needs tidying up, it calls the garbage collector. The garbage collector runs through variables currently in scope in your code, examining references to objects stored on the heap to identify which ones are accessible from your code — that is, which objects have references that refer to them. Any objects not referred to are deemed to be no longer accessible from your code and can therefore be removed. Java uses a system of garbage collection similar to this. Garbage collection works in .NET because IL has been designed to facilitate the process. The principle requires that you cannot get references to existing objects other than by copying existing references and that IL is type safe. In this context, if any reference to an object exists, there is sufficient information in the reference to exactly determine the type of the object. The garbage collection mechanism cannot be used with a language such as unmanaged C++, for example, because C++ enables pointers to be freely cast between types. One important aspect of garbage collection is that it is not deterministic. In other words, you cannot guarantee when the garbage collector will be called. It will be called when the CLR decides that it is needed; though you can override this process and call up the garbage collector in your code. Calling the garbage collector in your code is good for testing purposes, but you shouldn’t do this in a normal program. Look at Chapter 14, “Memory Management and Pointers,” for more information on the garbage collection process. Security .NET can excel in terms of complementing the security mechanisms provided by Windows because it can offer Code Access Security, whereas Windows offers only role-based security. Role-based security is based on the identity of the account under which the process runs (that is, who owns and runs the process). Code Access Security, by contrast, is based on what the code actually does and on how much the code is trusted. Because of the strong type safety of IL, the CLR can inspect code before running it to determine required security permissions. .NET also offers a mechanism by which code can indicate in advance what security permissions it requires to run. The importance of code-based security is that it reduces the risks associated with running code of dubious origin (such as code that you have downloaded from the Internet). For example, even if code runs under the administrator account, you can use code-based security to indicate that the code should still not be permitted to perform certain types of operations that the administrator account would normally be allowed to do, such as read or write to environment variables, read or write to the registry, or access the .NET reflection features. c01.indd 11 30-01-2014 19:56:53 12  ❘  CHAPTER 1  .NET Architecture NOTE  Security issues are covered in more depth in Chapter 22, “Security.” Application Domains Application domains are an important innovation in .NET and are designed to ease the overhead involved when running applications that need to be isolated from each other, but also need to communicate with each other. The classic example of this is a web server application, which may be simultaneously responding to a number of browser requests. It can, therefore, probably have a number of instances of the component responsible for servicing those requests running simultaneously. In pre-.NET days, the choice would be between allowing those instances to share a process (with the resultant risk of a problem in one running instance bringing the whole website down) or isolating those instances in separate processes (with the associated performance overhead). Before .NET, isolation of code was only possible by using different processes. When you start a new application, it runs within the context of a process. Windows isolates processes from each other through address spaces. The idea is that each process has 4GB of virtual memory available in which to store its data and executable code (4GB is for 32-bit systems; 64-bit systems use more memory). Windows imposes an extra level of indirection by which this virtual memory maps into a particular area of actual physical memory or disk space. Each process gets a different mapping, with no overlap between the actual physical memories that the blocks of virtual address space map to (see Figure 1-2). FIGURE 1-2 Physical memory or disk space PROCESS 1 4GB virtual memory Physical Memory Physical memory or disk space PROCESS 2 4GB virtual memory Self-describing Types In general, any process can access memory only by specifying an address in virtual memory — processes do not have direct access to physical memory. Hence, it is simply impossible for one process to access the memory allocated to another process. This provides an excellent guarantee that any badly behaved code cannot damage anything outside of its own address space. Processes do not just serve as a way to isolate instances of running code from each other; they also form the unit to which security privileges and permissions are assigned. Each process has its own security token, which indicates to Windows precisely what operations that process is permitted to do. Although processes are great for security reasons, their big disadvantage is in the area of performance. Often, a number of processes can actually work together, and therefore need to communicate with each other. The obvious example of this is where a process calls up a COM component, which is an executable and therefore is required to run in its own process. The same thing happens in COM when surrogates c01.indd 12 30-01-2014 19:56:55 A Closer Look at Intermediate Language  ❘  13 are used. Because processes cannot share any memory, a complex marshaling process must be used to copy data between the processes. This results in a significant performance hit. If you need components to work together and do not want that performance hit, you must use DLL-based components and have everything running in the same address space — with the associated risk that a badly behaved component can bring everything else down. Application domains are designed as a way to separate components without resulting in the performance problems associated with passing data between processes. The idea is that any one process is divided into a number of application domains. Each application domain roughly corresponds to a single application, and each thread of execution can run in a particular application domain (see Figure 1-3). If different executables run in the same process space, then they clearly can easily share data because theoretically they can directly see each other’s data. However, although this is possible in principle, the CLR makes sure that this does not happen in practice by inspecting the code for each running application to ensure that the code cannot stray outside of its own data areas. This looks, at first, like an almost impossible task to pull off — after all, how can you tell what the program is going to do without actually running it? It is usually possible to do this because of the strong type safety of the IL. In most cases, unless code uses unsafe features such as pointers, the data types it uses ensures that memory is not accessed inappropriately. For example, .NET array types perform bounds checking to ensure that no out-of-bounds array operations are permitted. If a running application does need to communicate or share data with other applications running in different application domains, it must do so by calling on .NET’s remoting services. Code that has been verified to check that it cannot access data outside its application domain (other than through the explicit remoting mechanism) is memory type safe. Such code can safely be run alongside other type-safe code in different application domains within the same process. Error Handling with Exceptions The .NET Framework is designed to facilitate handling of error conditions using the same mechanism based on exceptions that is employed by Java and C++. C++ developers should note that because of IL’s stronger typing system, there is no performance penalty associated with the use of exceptions with IL in the way that there is in C++. Also, the finally block, which has long been on many C++ developers’ wish lists, is supported by .NET and by C#. Exceptions are covered in detail in Chapter 16, “Errors and Exceptions.” Briefly, the idea is that certain areas of code are designated as exception handler routines, with each one dealing with a particular error condition (for example, a file not being found, or being denied permission to perform some operation). These conditions can be defined as narrowly or as widely as you want. The exception architecture ensures that when an error condition occurs, execution can immediately jump to the exception handler routine that is most specifically geared to handle the exception condition in question. The architecture of exception handling also provides a convenient means to pass an object containing precise details of the exception condition to an exception-handling routine. This object might include an appropriate message for the user and details of exactly where in the code the exception was detected. Most exception-handling architecture, including the control of program flow when an exception occurs, is handled by the high-level languages (C#, Visual Basic 2013, C++), and is not supported by any special IL commands. C#, for example, handles exceptions using try{}, catch{}, and finally{} blocks of code. (For more details, see Chapter 16.) What .NET does do, however, is provide the infrastructure to enable compilers that target .NET to support exception handling. In particular, it provides a set of .NET classes that can represent the exceptions and FIGURE 1-3 PROCESS - 4GB virtual memory APPLICATION DOMAIN: an application uses some of this virtual memory APPLICATION DOMAIN: another application uses some of this virtual memory c01.indd 13 30-01-2014 19:56:56 14  ❘  CHAPTER 1  .NET Architecture the language interoperability to enable the thrown exception objects to be interpreted by the exception- handling code, regardless of what language the exception-handling code is written in. This language independence is absent from both the C++ and Java implementations of exception handling; although it is present to a limited extent in the COM mechanism for handling errors, which involves returning error codes from methods and passing error objects around. Because exceptions are handled consistently in different languages is a crucial aspect of facilitating multi-language development. Use of Attributes Attributes are familiar to developers who use C++ to write COM components (through their use in Microsoft’s COM Interface Definition Language [IDL]). The initial idea of an attribute was that it provided extra information concerning some item in the program that could be used by the compiler. Attributes are supported in .NET — and now by C++, C#, and Visual Basic 2013. What is, however, particularly innovative about attributes in .NET is that you can define your own custom attributes in your source code. These user-defined attributes will be placed with the metadata for the corresponding data types or methods. This can be useful for documentation purposes, in which they can be used with reflection technology to perform programming tasks based on attributes. In addition, in common with the .NET philosophy of language independence, attributes can be defined in source code in one language and read by code written in another language. NOTE  Chapter 15, “Reflection,” covers attributes. Assemblies An assembly is the logical unit that contains compiled code targeted at the .NET Framework. This chapter doesn’t cover assemblies in detail because they are covered thoroughly in Chapter 19, “Assemblies,” but following are the main points. An assembly is completely self-describing and is a logical rather than a physical unit, which means that it can be stored across more than one file. (Indeed, dynamic assemblies are stored in memory, not on file.) If an assembly is stored in more than one file, there will be one main file that contains the entry point and describes the other files in the assembly. The same assembly structure is used for both executable code and library code. The only difference is that an executable assembly contains a main program entry point, whereas a library assembly does not. An important characteristic of assemblies is that they contain metadata that describes the types and methods defined in the corresponding code. An assembly, however, also contains assembly metadata that describes the assembly. This assembly metadata, contained in an area known as the manifest, enables checks to be made on the version of the assembly and on its integrity. NOTE  ildasm, a Windows-based utility, can be used to inspect the contents of an assembly, including the manifest and metadata. ildasm is discussed in Chapter 19. Because an assembly contains program metadata means that applications or other assemblies that call up code in a given assembly do not need to refer to the registry, or to any other data source, to find out how to use that assembly. This is a significant break from the old COM way to do things, in which the GUIDs of the components and interfaces had to be obtained from the registry, and in some cases, the details of the methods and properties exposed would need to be read from a type library. c01.indd 14 30-01-2014 19:56:56 Assemblies  ❘  15 Having data spread out in up to three different locations meant there was the obvious risk of something getting out of synchronization, which would prevent other software from using the component successfully. With assemblies, there is no risk of this happening because all the metadata is stored with the program executable instructions. Even though assemblies are stored across several files, there are still no problems with data going out of synchronization. This is because the file that contains the assembly entry point also stores details of, and a hash of, the contents of the other files, which means that if one of the files is replaced, or in any way tampered with, this will almost certainly be detected and the assembly will refuse to load. Assemblies come in two types: private and shared assemblies. Private Assemblies Private assemblies are the simplest type. They normally ship with software and are intended to be used only with that software. The usual scenario in which you ship private assemblies is when you supply an application in the form of an executable and a number of libraries, where the libraries contain code that should be used only with that application. The system guarantees that private assemblies will not be used by other software because an application may load only private assemblies located in the same folder that the main executable is loaded in, or in a subfolder of it. Because you would normally expect that commercial software would always be installed in its own directory, there is no risk of one software package overwriting, modifying, or accidentally loading private assemblies intended for another package. And, because private assemblies can be used only by the software package that they are intended for, you have much more control over what software uses them. There is, therefore, less need to take security precautions because there is no risk, for example, of some other commercial software overwriting one of your assemblies with some new version of it (apart from software designed specifically to perform malicious damage). There are also no problems with name collisions. If classes in your private assembly happen to have the same name as classes in someone else’s private assembly, that does not matter because any given application can see only the one set of private assemblies. Because a private assembly is entirely self-contained, the process to deploy it is simple. You simply place the appropriate file(s) in the appropriate folder in the file system. (No registry entries need to be made.) This process is known as zero impact (xcopy) installation. Shared Assemblies Shared assemblies are intended to be common libraries that any other application can use. Because any other software can access a shared assembly, more precautions need to be taken against the following risks: ➤➤ Name collisions, where another company’s shared assembly implements types that have the same names as those in your shared assembly. Because client code can theoretically have access to both assemblies simultaneously, this could be a serious problem. ➤➤ The risk of an assembly being overwritten by a different version of the same assembly — the new version is incompatible with some existing client code. The solution to these problems is placing shared assemblies in a special directory subtree in the file system, known as the global assembly cache (GAC). Unlike with private assemblies, this cannot be done by simply copying the assembly into the appropriate folder; it must be specifically installed into the cache. This process can be performed by a number of .NET utilities and requires certain checks on the assembly, as well as setting up of a small folder hierarchy within the assembly cache used to ensure assembly integrity. To prevent name collisions, shared assemblies are given a name based on private key cryptography. (Private assemblies are simply given the same name as their main filename.) This name is known as a strong name; it is guaranteed to be unique and must be quoted by applications that reference a shared assembly. Problems associated with the risk of overwriting an assembly are addressed by specifying version information in the assembly manifest and by allowing side-by-side installations. c01.indd 15 30-01-2014 19:56:57 16  ❘  CHAPTER 1  .NET Architecture Reflection Because assemblies store metadata, including details of all the types and members of these types defined in the assembly, you can access this metadata programmatically. Full details of this are given in Chapter 15. This technique, known as reflection, raises interesting possibilities because it means that managed code can actually examine other managed code, and can even examine itself, to determine information about that code. This is most commonly used to obtain the details of attributes; although you can also use reflection, among other purposes, as an indirect way to instantiate classes or calling methods, given the names of those classes or methods as strings. In this way, you could select classes to instantiate methods to call at runtime, rather than at compile time, based on user input (dynamic binding). Parallel Programming The .NET Framework enables you to take advantage of all the multicore processors available today. The parallel computing capabilities provide the means to separate work actions and run these across multiple processors. The parallel programming APIs available now make writing safe multithreaded code simple; though you must realize that you still need to account for race conditions and things such as deadlocks. The new parallel programming capabilities provide a new Task Parallel Library and a PLINQ Execution Engine. Chapter 21, “Tasks, Threads, and Synchronization,” covers parallel programming. Asynchronous Programming Based on the Task from the Task Parallel Library are the new async features of C# 5. Since .NET 1.0, many classes from the .NET Framework offered asynchronous methods besides the synchronous variant. The user interface thread should not be blocked when doing a task that takes a while. You’ve probably seen several programs that have become unresponsive, which is annoying. A problem with the asynchronous methods was that they were difficult to use. The synchronous variant was a lot easier to program with, and thus this one was usually used. Using the mouse the user is — with many years of experience — used to a delay. When moving objects or just using the scrollbar, a delay is normal. With new touch interfaces, if there’s a delay the experience for the user can be extremely annoying. This can be solved by calling asynchronous methods. If a method with the WinRT might take more than 50 milliseconds, the WinRT offers only asynchronous method calls. C# 5 now makes it easy to invoke new asynchronous methods. C# 5 defines two new keywords: async and await. These keywords and how they are used are discussed in Chapter 13, “Asynchronous Programming.” .NET Framework Classes Perhaps one of the biggest benefits to writing managed code, at least from a developer’s point of view, is that you can use the .NET base class library. The .NET base classes are a massive collection of managed code classes that enable you to do almost any of the tasks that were previously available through the Windows API. These classes follow the same object model that IL uses, based on single inheritance. This means that you can either instantiate objects of whichever .NET base class is appropriate or derive your own classes from them. The great thing about the .NET base classes is that they have been designed to be intuitive and easy to use. For example, to start a thread, you call the Start() method of the Thread class. To disable a TextBox, you set the Enabled property of a TextBox object to false. This approach — though familiar to Visual Basic and Java developers whose respective libraries are just as easy to use — will be a welcome relief to C++ developers, who for years have had to cope with such API functions as GetDIBits(), RegisterWndClassEx(), and IsEqualIID(), and a plethora of functions that require Windows handles to be passed around. However, C++ developers always had easy access to the entire Windows API, unlike Visual Basic 6 and Java developers who were more restricted in terms of the basic operating system functionality that they have c01.indd 16 30-01-2014 19:56:57 Namespaces  ❘  17 access to from their respective languages. What is new about the .NET base classes is that they combine the ease of use that was typical of the Visual Basic and Java libraries with the relatively comprehensive coverage of the Windows API functions. Many features of Windows are still not available through the base classes, and for those you need to call into the API functions, but in general, these are now confined to the more exotic features. For everyday use, you can probably find the base classes adequate. Moreover, if you do need to call into an API function, .NET offers a platform-invoke that ensures data types are correctly converted, so the task is no harder than calling the function directly from C++ code would have been — regardless of whether you code in C#, C++, or Visual Basic 2013. Although Chapter 3 is nominally dedicated to the subject of base classes, after you have completed the coverage of the syntax of the C# language, most of the rest of this book shows you how to use various classes within the .NET base class library for the .NET Framework 4.5. That is how comprehensive base classes are. As a rough guide, the areas covered by the .NET 4.5 base classes include the following: ➤➤ Core features provided by IL (including the primitive data types in the CTS discussed in Chapter 2) ➤➤ Windows UI support and controls (see Chapters 35–39) ➤➤ ASP.NET with Web Forms and MVC (see Chapters 30–42) ➤➤ Data access with ADO.NET and XML (see Chapters 32–34) ➤➤ File system and registry access (see Chapter 24, “Manipulating Files and Registry”) ➤➤ Networking and web browsing (see Chapter 26, “Networking”) ➤➤ .NET attributes and reflection (see Chapter 15) ➤➤ COM interoperability (see Chapter 23) Incidentally, according to Microsoft sources, a large proportion of the .NET base classes have actually been written in C#. Namespaces Namespaces are the way that .NET avoids name clashes between classes. They are designed to prevent situations in which you define a class to represent a customer, name your class Customer, and then someone else does the same thing. (A likely scenario in which — the proportion of businesses that have customers seems to be quite high.) A namespace is no more than a grouping of data types, but it has the effect that the names of all data types within a namespace are automatically prefixed with the name of the namespace. It is also possible to nest namespaces within each other. For example, most of the general-purpose .NET base classes are in a namespace called System. The base class Array is in this namespace, so its full name is System.Array. .NET requires all types to be defined in a namespace; for example, you could place your Customer class in a namespace called YourCompanyName.ProjectName. This class would have the full name YourCompanyName.ProjectName.Customer. NOTE  If a namespace is not explicitly supplied, the type will be added to a nameless global namespace. Microsoft recommends that for most purposes you supply at least two nested namespace names: the first one represents the name of your company, and the second one represents the name of the technology or software package of which the class is a member, such as YourCompanyName.SalesServices.Customer. This protects, in most situations, the classes in your application from possible name clashes with classes written by other organizations. Chapter 2 looks more closely at namespaces. c01.indd 17 30-01-2014 19:56:57 18  ❘  CHAPTER 1  .NET Architecture Creating .NET Applications Using C# You can also use C# to create console applications: text-only applications that run in a DOS window. You can probably use console applications when unit testing class libraries and for creating UNIX or Linux daemon processes. More often, however, you can use C# to create applications that use many of the technologies associated with .NET. This section gives you an overview of the different types of applications that you can write in C#. Creating ASP.NET Applications The original introduction of ASP.NET 1.0 fundamentally changed the web programming model. ASP.NET 4.5 is a major release of the product and builds upon its earlier achievements. ASP.NET 4.5 follows on a series of major revolutionary steps designed to increase your productivity. The primary goal of ASP.NET is to enable you to build powerful, secure, dynamic applications using the least possible amount of code. As this is a C# book, there are many chapters showing you how to use this language to build the latest in web applications. The following section explores the key features of ASP.NET. For more details, refer to Chapters 40 to 42. Features of ASP.NET With the invention of ASP.NET, there were only ASP.NET Web Forms, which had the goal of easily creating web applications in a way a Windows application developer was used to writing applications. It was the goal not to need to write HTML and JavaScript. Nowadays this is different again. HTML and JavaScript became important and modern again. And there’s a new ASP.NET Framework that makes it easy to do this and gives a separation based on the well-known Model View Controller (MVC) pattern for easier unit testing: ASP.NET MVC. ASP.NET was refactored to have a foundation available both for ASP.NET Web Forms and ASP.NET MVC, and then the UI frameworks are based on this foundation. NOTE  Chapter 40, “Core ASP.NET” covers the foundation of ASP.NET. ASP.NET Web Forms To make web page construction easy, Visual Studio 2013 supplies Web Forms. Web pages can be built graphically by dragging controls from a toolbox onto a form and then flipping over to the code aspect of that form and writing event handlers for the controls. When you use C# to create a Web Form, you create a C# class that inherits from the Page base class and an ASP.NET page that designates that class as its code-behind. Of course, you do not need to use C# to create a Web Form; you can use Visual Basic 2013 or another .NET-compliant language just as well. ASP.NET Web Forms provide a rich functionality with controls that do not create only simple HTML code, but with controls that do input validation using both JavaScript and server-side validation logic, grids, data sources to access the database, offer Ajax features for dynamically rendering just parts of the page on the client, and much more. NOTE  Chapter 41, “ASP.NET Web Forms” discusses ASP.NET Web Forms. Web Server Controls The controls used to populate a Web Form are not controls in the same sense as ActiveX controls. Rather, they are XML tags in the ASP.NET namespace that the web browser dynamically transforms into HTML and client-side script when a page is requested. Amazingly, the web server can render the same server-side c01.indd 18 30-01-2014 19:56:57 Creating .NET Applications Using C#  ❘  19 control in different ways, producing a transformation appropriate to the requestor’s particular web browser. This means that it is now easy to write fairly sophisticated user interfaces for web pages, without worrying about how to ensure that your page can run on any of the available browsers — because Web Forms take care of that for you. You can use C# or Visual Basic 2013 to expand the Web Form toolbox. Creating a new server-side control is simply a matter of implementing .NET’s System.Web.UI.WebControls.WebControl class. A SP.NE T MVC Visual Studio comes with ASP.NET MVC 4. This technology is already available in version 4. Contrary to Web Forms where HTML and JavaScript is abstracted away from the developer, with the advent of HTML 5 and jQuery, using these technologies has become more important again. With ASP.NET MVC the focus is on writing server-side code separated within model and controller and using views with just a little bit of server-side code to get information from the controller. This separation makes unit testing a lot easier and gives the full power to use HTML 5 and JavaScript libraries. NOTE  Chapter 42, “ASP.NET MVC” covers ASP.NET MVC. Windows Presentation Foundation (WPF) For creating Windows desktop applications, two technologies are available: Windows Forms and Windows Presentation Foundation. Windows Forms consists of classes that just wrap native Windows controls and is thus based on pixel graphics. Windows Presentation Foundation (WPF) is the newer technology based on vector graphics. WPF makes use of XAML in building applications. XAML stands for eXtensible Application Markup Language. This new way to create applications within a Microsoft environment is something introduced in 2006 and is part of the .NET Framework 3.0. This means that to run any WPF application, you need to make sure that at least the .NET Framework 3.0 is installed on the client machine. Of course, you get new WPF features with newer versions of the framework. With version 4.5, for example, the ribbon control and live shaping are new features among many new controls. XAML is the XML declaration used to create a form that represents all the visual aspects and behaviors of the WPF application. Though you can work with a WPF application programmatically, WPF is a step in the direction of declarative programming, which the industry is moving to. Declarative programming means that instead of creating objects through programming in a compiled language such as C#, VB, or Java, you declare everything through XML-type programming. Chapter 29, “Core XAML,” introduces XAML (which is also used with XML Paper Specification, Windows Workflow Foundation, and Windows Communication Foundation). Chapter 35, “Core WPF,” details how to build WPF applications using XAML and C#. Chapter 36, “Business Applications with WPF,” goes into more details on data-driven business applications with WPF and XAML. Printing and creating documents is another important aspect of WPF covered in Chapter 37, “Creating Documents with WPF.” Windows Store Apps Windows 8 started a new paradigm with touch-first Windows Store apps. With desktop applications the user usually gets a menu and a toolbar, and receives a chrome with the application to see what he can do next. Windows Store apps have the focus on the content. Chrome should be minimized to tasks the user can do with the content, and not on different options he has. The focus is on the current task, and not what the user might do next. This way the user remembers the application based on its content. Content and no chrome is a buzz phrase with this technology. c01.indd 19 30-01-2014 19:56:57 20  ❘  CHAPTER 1  .NET Architecture Windows Store apps can be written with C# and XAML, using the Windows Runtime with a subset of the .NET Framework. Windows Store apps offer huge new opportunities. The major disadvantage is that they are only available with Windows 8 and newer Windows operating systems. NOTE  Chapter 31, “Windows Runtime”, Chapter 38, “Windows Store Apps: UI,” and Chapter 39, “Windows Store Apps: Contracts and Devices,” cover creating Windows Store apps. Windows Services A Windows Service (originally called an NT Service) is a program designed to run in the background in Windows NT kernel based operating systems. Services are useful when you want a program to run continuously and ready to respond to events without having been explicitly started by the user. A good example is the World Wide Web Service on web servers, which listens for web requests from clients. It is easy to write services in C#. .NET Framework base classes are available in the System.ServiceProcess namespace that handles many of the boilerplate tasks associated with services. In addition, Visual Studio .NET enables you to create a C# Windows Service project, which uses C# source code for a basic Windows Service. Chapter 27, “Windows Services,” explores how to write C# Windows Services. Windows Communication Foundation One communication technology fused between client and server is the ASP.NET Web API. The ASP.NET Web API is easy to use but doesn’t offer a lot of features such as offered from the SOAP protocol. Windows Communication Foundation (WCF) is a feature-rich technology to offer a broad set of communication options. With WCF you can use a REST-based communication but also a SOAP-based communication with all the features used by standards-based web services such as security, transactions, duplex and one-way communication, routing, discovery, and so on. WCF provides you with the ability to build your service one time and then expose this service in a multitude of ways (under different protocols even) by just making changes within a configuration file. You can find that WCF is a powerful new way to connect disparate systems. Chapter 43, “Windows Communication Foundation,” covers this in detail. You can also find WCF-based technologies such as Message Queuing with WCF in Chapter 47, “Message Queuing.” ASP.NET Web API A new way for simple communication to occur between the client and the server — a REST-based style — is offered with the ASP.NET Web API. This new framework is based on ASP.NET MVC and makes use of controllers and routing. The client can receive JSON or Atom data based on the Open Data specification. The features of this new API make it easy to consume from web clients using JavaScript and also from Windows Store apps. NOTE  The ASP.NET Web API is covered in Chapter 44, “ASP.NET Web API.” Windows Workflow Foundation The Windows Workflow Foundation (WF) was introduced with the release of the .NET Framework 3.0 but had a good overhaul that many find more approachable now since .NET 4. There are some smaller improvements with .NET 4.5 as well. You can find that Visual Studio 2013 has greatly improved for working with WF and makes it easier to construct your workflows and write expressions using C# (instead of VB in the previous edition). You can also find a new state machine designer and new activities. c01.indd 20 30-01-2014 19:56:57 The Role of C# in the .NET Enterprise Architecture  ❘  21 NOTE  WF is covered in Chapter 45, “Windows Workflow Foundation.” The Role of C# in the .NET Enterprise Architecture New technologies are coming at a fast pace. What should you use for enterprise applications? There are many aspects that influence the decision. For example, what about the existing applications that have been developed with current technology knowledge of the developers. Can you integrate new features with legacy applications? Depending on the maintenance required, maybe it makes sense to rebuild some existing applications for easier use of new features. Usually, legacy and new can coexist for many years to come. What is the requirement for the client systems? Can the .NET Framework be upgraded to version 4.5, or is 2.0 a requirement? Or is .NET not available on the client? There are many decisions to make, and .NET gives many options. You can use .NET on the client with Windows Forms, WPF, or Windows 8-style apps. You can use .NET on the web server hosted with IIS and the ASP.NET Runtime with ASP.NET Web Forms or ASP.NET MVC. Services can run within IIS, and you can host the services from within Windows Services. C# presents an outstanding opportunity for organizations interested in building robust, n-tiered client-server applications. When combined with ADO.NET, C# has the capability to quickly and generically access data stores such as SQL Server or other databases with data providers. The ADO.NET Entity Framework can be an easy way to map database relations to object hierarchies. This is not only possible with SQL Server, but also many different databases where an Entity Framework provider is offered. The returned data can be easily manipulated using the ADO.NET object model or LINQ and automatically rendered as XML or JSON for transport across an office intranet. After a database schema has been established for a new project, C# presents an excellent medium for implementing a layer of data access objects, each of which could provide insertion, updates, and deletion access to a different database table. Because it’s the first component-based C language, C# is a great language for implementing a business object tier, too. It encapsulates the messy plumbing for intercomponent communication, leaving developers free to focus on gluing their data access objects together in methods that accurately enforce their organizations’ business rules. To create an enterprise application with C#, you create a class library project for the data access objects and another for the business objects. While developing, you can use Console projects to test the methods on your classes. Fans of extreme programming can build Console projects that can be executed automatically from batch files to unit test that working code has not been broken. On a related note, C# and .NET will probably influence the way you physically package your reusable classes. In the past, many developers crammed a multitude of classes into a single physical component because this arrangement made deployment a lot easier; if there were a versioning problem, you knew just where to look. Because deploying .NET components involves simply copying files into directories, developers can now package their classes into more logical, discrete components without encountering “DLL Hell.” Last, but not least, ASP.NET pages coded in C# constitute an excellent medium for user interfaces. Because ASP.NET pages compile, they execute quickly. Because they can be debugged in the Visual Studio 2013 IDE, they are robust. Because they support full-scale language features such as early binding, inheritance, and modularization, ASP.NET pages coded in C# are tidy and easily maintained. After the hype of SOA and service-based programming, nowadays using services has becoming the norm. The new hype is cloud-based programming, with Windows Azure as Microsoft’s offering. You can run .NET applications in a range from ASP.NET Web Forms, ASP.NET Web API, or WCF either on on-premise servers or in the cloud. Clients can make use of HTML 5 for a broad reach or make use of WPF or Windows Store apps for rich functionality. Still with new technologies and options, .NET has a prosperous life. c01.indd 21 30-01-2014 19:56:58 22  ❘  CHAPTER 1  .NET Architecture Summary This chapter covered a lot of ground, briefly reviewing important aspects of the .NET Framework and C#’s relationship to it. It started by discussing how all languages that target .NET are compiled into Microsoft Intermediate Language (IL) before this is compiled and executed by the Common Language Runtime (CLR). This chapter also discussed the roles of the following features of .NET in the compilation and execution process: ➤➤ Assemblies and .NET base classes ➤➤ COM components ➤➤ JIT compilation ➤➤ Application domains ➤➤ Garbage collection Figure 1-4 provides an overview of how these features come into play during compilation and execution. FIGURE 1-4 ASSEMBLY containing IL CODE COMPILATION EXECUTION Language Interoperability through CTS and CLS VB.NET Source Code .NET base classes Assemblies loaded CLR ORGANIZES: C# Source Code ASSEMBLY containing IL CODE JIT compilation Security permissions granted Memory type safety checked Creates App Domain Garbage collector cleans up sources PROCESS Application domain CODE EXECUTES HERE COM interop services legacy COM component You learned about the characteristics of IL, particularly its strong data typing and object orientation, and how these characteristics influence the languages that target .NET, including C#. You also learned how the strongly typed nature of IL enables language interoperability, as well as CLR services such as garbage collection and security. There was also a focus on the Common Language Specification (CLS) and the Common Type System (CTS) to help deal with language interoperability. Finally, you learned how C# can be used as the basis for applications built on several .NET technologies, including ASP.NET and WPF. Chapter 2 discusses how to write code in C#. c01.indd 22 30-01-2014 19:56:59 Core C# wHAT’s in THis CHAPTER? ➤➤ Declaring variables ➤➤ Initialization and scope of variables ➤➤ Predefi ned C# data types ➤➤ Dictating execution fl ow within a C# program using conditional statements, loops, and jump statements ➤➤ Enumerations ➤➤ Namespaces ➤➤ The Main() method ➤➤ Basic command-line C# compiler options ➤➤ Using System.Console to perform console I/O ➤➤ Using internal comments and documentation features ➤➤ Preprocessor directives ➤➤ Guidelines and conventions for good programming in C# wRox.Com CodE downloAds FoR THis CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ ArgsExample.cs ➤➤ DoubleMain.cs ➤➤ ElseIf.cs ➤➤ First.cs ➤➤ MathClient.cs ➤➤ MathLibrary.cs ➤➤ NestedFor.cs ➤➤ Scope.cs 2 c02.indd 23 30-01-2014 20:04:48 24  ❘  CHAPTER 2  Core C# ➤➤ ScopeBad.cs ➤➤ ScopeTest2.cs ➤➤ StringExample.cs ➤➤ Var.cs Fundamental C# Now that you understand more about what C# can do, you will want to learn how to use it. This chapter gives you a good start in that direction by providing a basic understanding of the fundamentals of C# programming, which is built on in subsequent chapters. By the end of this chapter, you will know enough C# to write simple programs (though without using inheritance or other object-oriented features, which are covered in later chapters). Your First C# Program Let’s start by compiling and running the simplest possible C# program — a simple console app consisting of a class that writes a message to the screen. Note  Later chapters present a number of code samples. The most common technique for writing C# programs is to use Visual Studio 2012 to generate a basic project and add your own code to it. However, because the aim of Part I is to teach the C# language, we are going to keep things simple and avoid relying on Visual Studio 2012 until Chapter 17, “Visual Studio 2012.” Instead, we present the code as simple files that you can type in using any text editor and compile from the command line. The Code Type the following into a text editor (such as Notepad), and save it with a .cs extension (for example, First.cs). The Main() method is shown here (for more information, see “The Main Method” section later in this chapter): using System; namespace Wrox { public class MyFirstClass { static void Main() { Console.WriteLine("Hello from Wrox."); Console.ReadLine(); return; } } } Compiling and Running the Program You can compile this program by simply running the C# command-line compiler (csc.exe) against the source file, like this: csc First.cs c02.indd 24 30-01-2014 20:04:48 Your First C# Program  ❘  25 If you want to compile code from the command line using the csc command, you should be aware that the .NET command-line tools, including csc, are available only if certain environment variables have been set up. Depending on how you installed .NET (and Visual Studio), this may or may not be the case on your machine. Note  If you do not have the environment variables set up, you have two options: The first is to run the batch file %Microsoft Visual Studio 2013%\Common7\Tools\ vsvars32.bat from the command prompt before running csc, where %Microsoft Visual Studio 2013% is the folder to which Visual Studio 2013 has been installed. The second, and easier, way is to use the Visual Studio 2013 command prompt instead of the usual command prompt window. To find the Visual Studio 2013 command prompt from the Start menu, select Programs ➪ Microsoft Visual Studio 2013 ➪Visual Studio Tools. It is simply a command prompt window that automatically runs vsvars32.bat when it opens. Compiling the code produces an executable file named First.exe, which you can run from the command line or from Windows Explorer like any other executable. Give it a try: csc First.cs Microsoft (R) Visual C# Compiler version 12.0.21005.1 for C# 5.0 Copyright (C) Microsoft Corporation. All rights reserved. First.exe Hello from Wrox. A Closer Look First, a few general comments about C# syntax. In C#, as in other C-style languages, most statements end in a semicolon (;) and can continue over multiple lines without needing a continuation character. Statements can be joined into blocks using curly braces ({}). Single-line comments begin with two forward slash characters (//), and multiline comments begin with a slash and an asterisk (/*) and end with the same combination reversed (*/). In these aspects, C# is identical to C++ and Java but different from Visual Basic. It is the semicolons and curly braces that give C# code such a different visual appearance from Visual Basic code. If your background is predominantly Visual Basic, take extra care to remember the semicolon at the end of every statement. Omitting this is usually the biggest single cause of compilation errors among developers new to C-style languages. Another thing to remember is that C# is case sensitive. That means the variables named myVar and MyVar are two different variables. The first few lines in the previous code example are related to namespaces (mentioned later in this chapter), which is a way to group together associated classes. The namespace keyword declares the namespace with which your class should be associated. All code within the braces that follow it is regarded as being within that namespace. The using statement specifies a namespace that the compiler should look at to find any classes that are referenced in your code but aren’t defined in the current namespace. This serves the same purpose as the import statement in Java and the using namespace statement in C++. using System; namespace Wrox { The reason for the presence of the using statement in the First.cs file is that you are going to use a library class, System.Console. The using System statement enables you to refer to this class simply as Console (and similarly for any other classes in the System namespace). Without using, you would have to fully qualify the call to the Console.WriteLine method like this: System.Console.WriteLine("Hello from Wrox."); c02.indd 25 30-01-2014 20:04:49 26  ❘  CHAPTER 2  Core C# The standard System namespace is where the most commonly used .NET types reside. It is important to realize that everything you do in C# depends on the .NET base classes. In this case, you are using the Console class within the System namespace to write to the console window. C# has no built-in keywords of its own for input or output; it is completely reliant on the .NET classes. Note  Because almost every C# program uses classes in the System namespace, we will assume that a using System; statement is present in the file for all code snippets in this chapter. Next, you declare a class called MyFirstClass. However, because it has been placed in a namespace called Wrox, the fully qualified name of this class is Wrox.MyFirstCSharpClass: class MyFirstCSharpClass { All C# code must be contained within a class. The class declaration consists of the class keyword, followed by the class name and a pair of curly braces. All code associated with the class should be placed between these braces. Next, you declare a method called Main(). Every C# executable (such as console applications, Windows applications, and Windows services) must have an entry point — the Main() method (note the capital M): public static void Main() { The method is called when the program is started. This method must return either nothing (void) or an integer (int). Note the format of method definitions in C#: [modifiers] return_type MethodName([parameters]) { // Method body. NB. This code block is pseudo-code. } Here, the first square brackets represent certain optional keywords. Modifiers are used to specify certain features of the method you are defining, such as from where the method can be called. In this case, you have two modifiers: public and static. The public modifier means that the method can be accessed from anywhere, so it can be called from outside your class. The static modifier indicates that the method does not operate on a specific instance of your class and therefore is called without first instantiating the class. This is important because you are creating an executable rather than a class library. You set the return type to void, and in the example you don’t include any parameters. Finally, we come to the code statements themselves: Console.WriteLine("Hello from Wrox."); Console.ReadLine(); return; In this case, you simply call the WriteLine() method of the System.Console class to write a line of text to the console window. WriteLine() is a static method, so you don’t need to instantiate a Console object before calling it. Console.ReadLine() reads user input. Adding this line forces the application to wait for the carriage-return key to be pressed before the application exits, and, in the case of Visual Studio 2013, the console window disappears. You then call return to exit from the method (also, because this is the Main() method, you exit the program as well). You specified void in your method header, so you don’t return any values. Now that you have had a taste of basic C# syntax, you are ready for more detail. Because it is virtually impossible to write any nontrivial program without variables, we will start by looking at variables in C#. c02.indd 26 30-01-2014 20:04:49 Variables  ❘  27 Variables You declare variables in C# using the following syntax: datatype identifier; For example: int i; This statement declares an int named i. The compiler won’t actually let you use this variable in an expression until you have initialized it with a value. After it has been declared, you can assign a value to the variable using the assignment operator, =: i = 10; You can also declare the variable and initialize its value at the same time: int i = 10; If you declare and initialize more than one variable in a single statement, all the variables will be of the same data type: int x = 10, y =20; // x and y are both ints To declare variables of different types, you need to use separate statements. You cannot assign different data types within a multiple-variable declaration: int x = 10; bool y = true; // Creates a variable that stores true or false int x = 10, bool y = true; // This won't compile! Notice the // and the text after it in the preceding examples. These are comments. The // character sequence tells the compiler to ignore the text that follows on this line because it is included for a human to better understand the program, not part of the program itself. We further explain comments in code later in this chapter. Initialization of Variables Variable initialization demonstrates an example of C#’s emphasis on safety. Briefly, the C# compiler requires that any variable be initialized with some starting value before you refer to that variable in an operation. Most modern compilers will flag violations of this as a warning, but the ever-vigilant C# compiler treats such violations as errors. This prevents you from unintentionally retrieving junk values from memory left over from other programs. C# has two methods for ensuring that variables are initialized before use: ➤➤ Variables that are fields in a class or struct, if not initialized explicitly, are by default zeroed out when they are created (classes and structs are discussed later). ➤➤ Variables that are local to a method must be explicitly initialized in your code prior to any statements in which their values are used. In this case, the initialization doesn’t have to happen when the variable is declared, but the compiler checks all possible paths through the method and flags an error if it detects any possibility of the value of a local variable being used before it is initialized. For example, you can’t do the following in C#: public static int Main() { int d; Console.WriteLine(d); // Can't do this! Need to initialize d before use return 0; } c02.indd 27 30-01-2014 20:04:49 28  ❘  CHAPTER 2  Core C# Notice that this code snippet demonstrates defining Main() so that it returns an int instead of void. If you attempt to compile the preceding lines, you will receive this error message: Use of unassigned local variable 'd' Consider the following statement: Something objSomething; In C#, this line of code would create only a reference for a Something object, but this reference would not yet actually refer to any object. Any attempt to call a method or property against this variable would result in an error. Instantiating a reference object in C# requires use of the new keyword. You create a reference as shown in the previous example and then point the reference at an object allocated on the heap using the new keyword: objSomething = new Something(); // This creates a Something on the heap Type Inference Type inference makes use of the var keyword. The syntax for declaring the variable changes somewhat. The compiler “infers” what the type of the variable is by what the variable is initialized to. For example: int someNumber = 0; becomes: var someNumber = 0; Even though someNumber is never declared as being an int, the compiler figures this out and someNumber is an int for as long as it is in scope. Once compiled, the two preceding statements are equal. Here is a short program to demonstrate: using System; namespace Wrox { class Program { static void Main(string[] args) { var name = "Bugs Bunny"; var age = 25; var isRabbit = true; Type nameType = name.GetType(); Type ageType = age.GetType(); Type isRabbitType = isRabbit.GetType(); Console.WriteLine("name is type " + nameType.ToString()); Console.WriteLine("age is type " + ageType.ToString()); Console.WriteLine("isRabbit is type " + isRabbitType.ToString()); } } } The output from this program is as follows: name is type System.String age is type System.Int32 isRabbit is type System.Bool c02.indd 28 30-01-2014 20:04:49 Variables  ❘  29 There are a few rules that you need to follow: ➤➤ The variable must be initialized. Otherwise, the compiler doesn’t have anything from which to infer the type. ➤➤ The initializer cannot be null. ➤➤ The initializer must be an expression. ➤➤ You can’t set the initializer to an object unless you create a new object in the initializer. We examine this more closely in the discussion of anonymous types in Chapter 3, “Objects and Types.” After the variable has been declared and the type inferred, the variable’s type cannot be changed. When established, the variable’s type follows all the strong typing rules that any other variable type must follow. Variable Scope The scope of a variable is the region of code from which the variable can be accessed. In general, the scope is determined by the following rules: ➤➤ A field (also known as a member variable) of a class is in scope for as long as its containing class is in scope. ➤➤ A local variable is in scope until a closing brace indicates the end of the block statement or method in which it was declared. ➤➤ A local variable that is declared in a for, while, or similar statement is in scope in the body of that loop. Scope Clashes for Local Variables It’s common in a large program to use the same variable name for different variables in different parts of the program. This is fine as long as the variables are scoped to completely different parts of the program so that there is no possibility for ambiguity. However, bear in mind that local variables with the same name can’t be declared twice in the same scope. For example, you can’t do this: int x = 20; // some more code int x = 30; Consider the following code sample: using System; namespace Wrox.ProCSharp.Basics { public class ScopeTest { public static int Main() { for (int i = 0; i < 10; i++) { Console.WriteLine(i); } // i goes out of scope here // We can declare a variable named i again, because // there's no other variable with that name in scope for (int i = 9; i >= 0; i -- ) { Console.WriteLine(i); } // i goes out of scope here. return 0; } } } c02.indd 29 11-03-2014 16:42:57 30  ❘  CHAPTER 2  Core C# This code simply prints out the numbers from 0 to 9, and then back again from 9 to 0, using two for loops. The important thing to note is that you declare the variable i twice in this code, within the same method. You can do this because i is declared in two separate loops, so each i variable is local to its own loop. Here’s another example: public static int Main() { int j = 20; for (int i = 0; i < 10; i++) { int j = 30; // Can't do this — j is still in scope Console.WriteLine(j + i); } return 0; } If you try to compile this, you’ll get an error like the following: ScopeTest.cs(12,15): error CS0136: A local variable named 'j' cannot be declared in this scope because it would give a different meaning to 'j', which is already used in a 'parent or current' scope to denote something else. This occurs because the variable j, which is defined before the start of the for loop, is still in scope within the for loop, and won’t go out of scope until the Main() method has finished executing. Although the second j (the illegal one) is in the loop’s scope, that scope is nested within the Main() method’s scope. The compiler has no way to distinguish between these two variables, so it won’t allow the second one to be declared. Scope Clashes for Fields and Local Variables In certain circumstances, however, you can distinguish between two identifiers with the same name (although not the same fully qualified name) and the same scope, and in this case the compiler allows you to declare the second variable. That’s because C# makes a fundamental distinction between variables that are declared at the type level (fields) and variables that are declared within methods (local variables). Consider the following code snippet: using System; namespace Wrox { class ScopeTest2 { static int j = 20; public static void Main() { int j = 30; Console.WriteLine(j); return; } } } This code will compile even though you have two variables named j in scope within the Main() method: the j that was defined at the class level, and doesn’t go out of scope until the class is destroyed (when the Main() method terminates and the program ends); and the j defined in Main(). In this case, the new variable named j that you declare in the Main() method hides the class-level variable with the same name, so when you run this code, the number 30 is displayed. What if you want to refer to the class-level variable? You can actually refer to fields of a class or struct from outside the object, using the syntax object.fieldname. In the previous example, you are accessing a static field (you’ll learn what this means in the next section) from a static method, so you can’t use an instance of the class; you just use the name of the class itself: c02.indd 30 30-01-2014 20:04:49 Predefined Data Types  ❘  31 .. public static void Main() { int j = 30; Console.WriteLine(j); Console.WriteLine(ScopeTest2.j); } .. If you were accessing an instance field (a field that belongs to a specific instance of the class), you would need to use the this keyword instead. Constants As the name implies, a constant is a variable whose value cannot be changed throughout its lifetime. Prefixing a variable with the const keyword when it is declared and initialized designates that variable as a constant: const int a = 100; // This value cannot be changed. Constants have the following characteristics: ➤➤ They must be initialized when they are declared; and after a value has been assigned, it can never be overwritten. ➤➤ The value of a constant must be computable at compile time. Therefore, you can’t initialize a constant with a value taken from a variable. If you need to do this, you must use a read-only field (this is explained in Chapter 3). ➤➤ Constants are always implicitly static. However, notice that you don’t have to (and, in fact, are not permitted to) include the static modifier in the constant declaration. At least three advantages exist for using constants in your programs: ➤➤ Constants make your programs easier to read by replacing magic numbers and strings with readable names whose values are easy to understand. ➤➤ Constants make your programs easier to modify. For example, assume that you have a SalesTax constant in one of your C# programs, and that constant is assigned a value of 6 percent. If the sales tax rate changes later, you can modify the behavior of all tax calculations simply by assigning a new value to the constant; you don’t have to hunt through your code for the value .06 and change each one, hoping you will find all of them. ➤➤ Constants help prevent mistakes in your programs. If you attempt to assign another value to a constant somewhere in your program other than at the point where the constant is declared, the compiler will flag the error. Predefined Data Types Now that you have seen how to declare variables and constants, let’s take a closer look at the data types available in C#. As you will see, C# is much stricter about the types available and their definitions than some other languages. Value Types and Reference Types Before examining the data types in C#, it is important to understand that C# distinguishes between two categories of data type: ➤➤ Value types ➤➤ Reference types c02.indd 31 30-01-2014 20:04:50 32  ❘  CHAPTER 2  Core C# The next few sections look in detail at the syntax for value and reference types. Conceptually, the difference is that a value type stores its value directly, whereas a reference type stores a reference to the value. These types are stored in different places in memory; value types are stored in an area known as the stack, and reference types are stored in an area known as the managed heap. It is important to be aware of whether a type is a value type or a reference type because of the different effect each assignment has. For example, int is a value type, which means that the following statement results in two locations in memory storing the value 20: // i and j are both of type int i = 20; j = i; However, consider the following example. For this code, assume you have defined a class called Vector; and that Vector is a reference type and has an int member variable called Value: Vector x, y; x = new Vector(); x.Value = 30; // Value is a field defined in Vector class y = x; Console.WriteLine(y.Value); y.Value = 50; Console.WriteLine(x.Value); The crucial point to understand is that after executing this code, there is only one Vector object: x and y both point to the memory location that contains this object. Because x and y are variables of a reference type, declaring each variable simply reserves a reference — it doesn’t instantiate an object of the given type. In neither case is an object actually created. To create an object, you have to use the new keyword, as shown. Because x and y refer to the same object, changes made to x will affect y and vice versa. Hence, the code will display 30 and then 50. Note  C++ developers should note that this syntax is like a reference, not a pointer. You use the . notation, not ->, to access object members. Syntactically, C# references look more like C++ reference variables. However, behind the superficial syntax, the real similarity is with C++ pointers. If a variable is a reference, it is possible to indicate that it does not refer to any object by setting its value to null: y = null; If a reference is set to null, then clearly it is not possible to call any nonstatic member functions or fields against it; doing so would cause an exception to be thrown at runtime. In C#, basic data types such as bool and long are value types. This means that if you declare a bool variable and assign it the value of another bool variable, you will have two separate bool values in memory. Later, if you change the value of the original bool variable, the value of the second bool variable does not change. These types are copied by value. In contrast, most of the more complex C# data types, including classes that you yourself declare, are reference types. They are allocated upon the heap, have lifetimes that can span multiple function calls, and can be accessed through one or several aliases. The Common Language Runtime (CLR) implements an elaborate algorithm to track which reference variables are still reachable and which have been orphaned. Periodically, the CLR will destroy orphaned objects and return the memory that they once occupied back to the operating system. This is done by the garbage collector. C# has been designed this way because high performance is best served by keeping primitive types (such as int and bool) as value types, and larger types that contain many fields (as is usually the case with classes) as reference types. If you want to define your own type as a value type, you should declare it as a struct. c02.indd 32 30-01-2014 20:04:50 Predefined Data Types  ❘  33 CTS Types As mentioned in Chapter 1, “.NET Architecture,” the basic predefined types recognized by C# are not intrinsic to the language but are part of the .NET Framework. For example, when you declare an int in C#, you are actually declaring an instance of a .NET struct, System.Int32. This may sound like a small point, but it has a profound significance: It means that you can treat all the primitive data types syntactically, as if they were classes that supported certain methods. For example, to convert an int i to a string, you can write the following: string s = i.ToString(); It should be emphasized that behind this syntactical convenience, the types really are stored as primitive types, so absolutely no performance cost is associated with the idea that the primitive types are notionally represented by .NET structs. The following sections review the types that are recognized as built-in types in C#. Each type is listed, along with its definition and the name of the corresponding .NET type (CTS type). C# has 15 predefined types, 13 value types, and 2 (string and object) reference types. Predefined Value Types The built-in CTS value types represent primitives, such as integer and floating-point numbers, character, and Boolean types. Integer Types C# supports eight predefined integer types, shown in the following table. Name CTS Type Description Range (min:max) sbyte System.SByte 8-bit signed integer -128:127 (-27:27–1) short System.Int16 16-bit signed integer -32,768:32,767 (-215:215–1) int System.Int32 32-bit signed integer -2,147,483,648:2,147,483,647 (-231:231–1) long System.Int64 64-bit signed integer -9,223,372,036,854,775,808: 9,223,372,036,854,775,807 (-263:263–1) byte System.Byte 8-bit unsigned integer 0:255 (0:28–1) ushort System.UInt16 16-bit unsigned integer 0:65,535 (0:216–1) uint System.UInt32 32-bit unsigned integer 0:4,294,967,295 (0:232–1) ulong System.UInt64 64-bit unsigned integer 0:18,446,744,073,709,551,615 (0:264–1) Some C# types have the same names as C++ and Java types but have different definitions. For example, in C# an int is always a 32-bit signed integer. In C++ an int is a signed integer, but the number of bits is platform-dependent (32 bits on Windows). In C#, all data types have been defined in a platform-independent manner to allow for the possible future porting of C# and .NET to other platforms. A byte is the standard 8-bit type for values in the range 0 to 255 inclusive. Be aware that, in keeping with its emphasis on type safety, C# regards the byte type and the char type as completely distinct, and any programmatic conversions between the two must be explicitly requested. Also be aware that unlike the other types in the integer family, a byte type is by default unsigned. Its signed version bears the special name sbyte. With .NET, a short is no longer quite so short; it is now 16 bits long. The int type is 32 bits long. The long type reserves 64 bits for values. All integer-type variables can be assigned values in decimal or hex notation. The latter requires the 0x prefix: long x = 0x12ab; c02.indd 33 30-01-2014 20:04:50 34  ❘  CHAPTER 2  Core C# If there is any ambiguity about whether an integer is int, uint, long, or ulong, it will default to an int. To specify which of the other integer types the value should take, you can append one of the following characters to the number: uint ui = 1234U; long l = 1234L; ulong ul = 1234UL; You can also use lowercase u and l, although the latter could be confused with the integer 1 (one). Floating-Point Types Although C# provides a plethora of integer data types, it supports floating-point types as well. Name CTS Type Description Significant Figures Range (Approximate) float System.Single 32-bit, single-precision floating point 7 ±1.5 × 10245 to ±3.4 × 1038 double System.Double 64-bit, double-precision floating point 15/16 ±5.0 × 102324 to ±1.7 × 10308 The float data type is for smaller floating-point values, for which less precision is required. The double data type is bulkier than the float data type but offers twice the precision (15 digits). If you hard-code a non-integer number (such as 12.3), the compiler will normally assume that you want the number interpreted as a double. To specify that the value is a float, append the character F (or f) to it: float f = 12.3F; The Decimal Type The decimal type represents higher-precision floating-point numbers, as shown in the following table. Name CTS Type Description Significant Figures Range (Approximate) decimal System.Decimal 128-bit, high-precision decimal notation 28 ±1.0 × 10228 to ± 7.9 × 1028 One of the great things about the CTS and C# is the provision of a dedicated decimal type for financial calculations. How you use the 28 digits that the decimal type provides is up to you. In other words, you can track smaller dollar amounts with greater accuracy for cents or larger dollar amounts with more rounding in the fractional portion. Bear in mind, however, that decimal is not implemented under the hood as a primitive type, so using decimal has a performance effect on your calculations. To specify that your number is a decimal type rather than a double, float, or an integer, you can append the M (or m) character to the value, as shown here: decimal d = 12.30M; The Boolean Type The C# bool type is used to contain Boolean values of either true or false. Name CTS Type Description Significant Figures Range (Approximate) bool System.Boolean Represents true or false NA true or false c02.indd 34 30-01-2014 20:04:50 Predefined Data Types  ❘  35 You cannot implicitly convert bool values to and from integer values. If a variable (or a function return type) is declared as a bool, you can only use values of true and false. You will get an error if you try to use zero for false and a nonzero value for true. The Character Type For storing the value of a single character, C# supports the char data type. Name CTS Type Values char System.Char Represents a single 16-bit (Unicode) character Literals of type char are signified by being enclosed in single quotation marks — for example, 'A'. If you try to enclose a character in double quotation marks, the compiler will treat this as a string and throw an error. As well as representing chars as character literals, you can represent them with four-digit hex Unicode values (for example, '\u0041'), as integer values with a cast (for example, (char)65), or as hexadecimal values (for example,'\x0041'). You can also represent them with an escape sequence, as shown in the following table. Escape Sequence Character \' Single quotation mark \" Double quotation mark \\ Backslash \0 Null \a Alert \b Backspace \f Form feed \n Newline \r Carriage return \t Tab character \v Vertical tab Predefined Reference Types C# supports two predefined reference types, object and string, described in the following table. Name CTS Type Description object System.Object The root type. All other types (including value types) in the CTS are derived from object. string System.String Unicode character string The object Type Many programming languages and class hierarchies provide a root type, from which all other objects in the hierarchy are derived. C# and .NET are no exception. In C#, the object type is the ultimate parent type from which all other intrinsic and user-defined types are derived. This means that you can use the object type for two purposes: ➤➤ You can use an object reference to bind to an object of any particular subtype. For example, in Chapter 7, “Operators and Casts,” you will see how you can use the object type to box a value object on the stack to move it to the heap; object references are also useful in reflection, when code must manipulate objects whose specific types are unknown. c02.indd 35 30-01-2014 20:04:51 36  ❘  CHAPTER 2  Core C# ➤➤ The object type implements a number of basic, general-purpose methods, which include Equals(), GetHashCode(), GetType(), and ToString(). Responsible user-defined classes may need to provide replacement implementations of some of these methods using an object-oriented technique known as overriding, which is discussed in Chapter 4, “Inheritance.” When you override ToString(), for example, you equip your class with a method for intelligently providing a string representation of itself. If you don’t provide your own implementations for these methods in your classes, the compiler will pick up the implementations in object, which may or may not be correct or sensible in the context of your classes. We examine the object type in more detail in subsequent chapters. The string Type C# recognizes the string keyword, which under the hood is translated to the .NET class, System.String. With it, operations like string concatenation and string copying are a snap: string str1 = "Hello "; string str2 = "World"; string str3 = str1 + str2; // string concatenation Despite this style of assignment, string is a reference type. Behind the scenes, a string object is allocated on the heap, not the stack; and when you assign one string variable to another string, you get two references to the same string in memory. However, string differs from the usual behavior for reference types. For example, strings are immutable. Making changes to one of these strings creates an entirely new string object, leaving the other string unchanged. Consider the following code: using System; class StringExample { public static int Main() { string s1 = "a string"; string s2 = s1; Console.WriteLine("s1 is " + s1); Console.WriteLine("s2 is " + s2); s1 = "another string"; Console.WriteLine("s1 is now " + s1); Console.WriteLine("s2 is now " + s2); return 0; } } The output from this is as follows: s1 is a string s2 is a string s1 is now another string s2 is now a string Changing the value of s1 has no effect on s2, contrary to what you’d expect with a reference type! What’s happening here is that when s1 is initialized with the value a string, a new string object is allocated on the heap. When s2 is initialized, the reference points to this same object, so s2 also has the value a string. However, when you now change the value of s1, instead of replacing the original value, a new object is allocated on the heap for the new value. The s2 variable will still point to the original object, so its value is unchanged. Under the hood, this happens as a result of operator overloading, a topic that is explored in Chapter 7. In general, the string class has been implemented so that its semantics follow what you would normally intuitively expect for a string. String literals are enclosed in double quotation marks ("."); if you attempt to enclose a string in single quotation marks, the compiler will take the value as a char and throw an error. C# strings can contain the same Unicode and hexadecimal escape sequences as chars. Because these escape sequences start with a backslash, you can’t use this character unescaped in a string. Instead, you need to escape it with two backslashes (\\): string filepath = "C:\\ProCSharp\\First.cs"; c02.indd 36 30-01-2014 20:04:51 Flow Control  ❘  37 Even if you are confident that you can remember to do this all the time, typing all those double backslashes can prove annoying. Fortunately, C# gives you an alternative. You can prefix a string literal with the at character (@) and all the characters after it will be treated at face value; they won’t be interpreted as escape sequences: string filepath = @"C:\ProCSharp\First.cs"; This even enables you to include line breaks in your string literals: string jabberwocky = @"'Twas brillig and the slithy toves Did gyre and gimble in the wabe."; In this case, the value of jabberwocky would be this: 'Twas brillig and the slithy toves Did gyre and gimble in the wabe. Flow Control This section looks at the real nuts and bolts of the language: the statements that allow you to control the flow of your program rather than execute every line of code in the order it appears in the program. Conditional Statements Conditional statements allow you to branch your code depending on whether certain conditions are met or the value of an expression. C# has two constructs for branching code: the if statement, which allows you to test whether a specific condition is met; and the switch statement, which allows you to compare an expression with several different values. The if Statement For conditional branching, C# inherits the C and C++ if.else construct. The syntax should be fairly intuitive for anyone who has done any programming with a procedural language: if (condition) statement(s) else statement(s) If more than one statement is to be executed as part of either condition, these statements need to be joined together into a block using curly braces ({.}). (This also applies to other C# constructs where statements can be joined into a block, such as the for and while loops): bool isZero; if (i == 0) { isZero = true; Console.WriteLine("i is Zero"); } else { isZero = false; Console.WriteLine("i is Non-zero"); } If you want to, you can use an if statement without a final else statement. You can also combine else if clauses to test for multiple conditions: using System; namespace Wrox { class MainEntryPoint { c02.indd 37 30-01-2014 20:04:51 38  ❘  CHAPTER 2  Core C# static void Main(string[] args) { Console.WriteLine("Type in a string"); string input; input = Console.ReadLine(); if (input == "") { Console.WriteLine("You typed in an empty string."); } else if (input.Length < 5) { Console.WriteLine("The string had less than 5 characters."); } else if (input.Length < 10) { Console.WriteLine("The string had at least 5 but less than 10 Characters."); } Console.WriteLine("The string was " + input); } } There is no limit to how many else ifs you can add to an if clause. Note that the previous example declares a string variable called input, gets the user to enter text at the command line, feeds this into input, and then tests the length of this string variable. The code also shows how easy string manipulation can be in C#. To find the length of input, for example, use input.Length. Another point to note about if is that you don’t need to use the braces if there’s only one statement in the conditional branch: if (i == 0) Let's add some brackets here. Console.WriteLine("i is Zero"); // This will only execute if i == 0 Console.WriteLine("i can be anything"); // Will execute whatever the // value of i However, for consistency, many programmers prefer to use curly braces whenever they use an if statement. The if statements presented also illustrate some of the C# operators that compare values. Note in particular that C# uses == to compare variables for equality. Do not use = for this purpose. A single = is used to assign values. In C#, the expression in the if clause must evaluate to a Boolean. It is not possible to test an integer directly (returned from a function, for example). You have to convert the integer that is returned to a Boolean true or false, for example, by comparing the value with zero or null: if (DoSomething() != 0) { // Non-zero value returned } else { // Returned zero } The switch Statement The switch / case statement is good for selecting one branch of execution from a set of mutually exclusive ones. It takes the form of a switch argument followed by a series of case clauses. When the expression in the switch argument evaluates to one of the values beside a case clause, the code immediately following the case clause executes. This is one example for which you don’t need to use curly braces to join statements into blocks; instead, you mark the end of the code for each case using the break statement. You can also c02.indd 38 30-01-2014 20:04:51 Flow Control  ❘  39 include a default case in the switch statement, which will execute if the expression evaluates to none of the other cases. The following switch statement tests the value of the integerA variable: switch (integerA) { case 1: Console.WriteLine("integerA =1"); break; case 2: Console.WriteLine("integerA =2"); break; case 3: Console.WriteLine("integerA =3"); break; default: Console.WriteLine("integerA is not 1,2, or 3"); break; } Note that the case values must be constant expressions; variables are not permitted. Though the switch.case statement should be familiar to C and C++ programmers, C#’s switch.case is a bit safer than its C++ equivalent. Specifically, it prohibits fall-through conditions in almost all cases. This means that if a case clause is fired early on in the block, later clauses cannot be fired unless you use a goto statement to indicate that you want them fired, too. The compiler enforces this restriction by flagging every case clause that is not equipped with a break statement as an error: Control cannot fall through from one case label ('case 2:') to another Although it is true that fall-through behavior is desirable in a limited number of situations, in the vast majority of cases it is unintended and results in a logical error that’s hard to spot. Isn’t it better to code for the norm rather than for the exception? By getting creative with goto statements, you can duplicate fall-through functionality in your switch. cases. However, if you find yourself really wanting to, you probably should reconsider your approach. The following code illustrates both how to use goto to simulate fall-through, and how messy the resultant code can be: // assume country and language are of type string switch(country) { case "America": CallAmericanOnlyMethod(); goto case "Britain"; case "France": language = "French"; break; case "Britain": language = "English"; break; } There is one exception to the no-fall-through rule, however, in that you can fall through from one case to the next if that case is empty. This allows you to treat two or more cases in an identical way (without the need for goto statements): switch(country) { case "au": case "uk": case "us": language = "English"; break; c02.indd 39 30-01-2014 20:04:51 40  ❘  CHAPTER 2  Core C# case "at": case "de": language = "German"; break; } One intriguing point about the switch statement in C# is that the order of the cases doesn’t matter — you can even put the default case first! As a result, no two cases can be the same. This includes different constants that have the same value, so you can’t, for example, do this: // assume country is of type string const string england = "uk"; const string britain = "uk"; switch(country) { case england: case britain: // This will cause a compilation error. language = "English"; break; } The previous code also shows another way in which the switch statement is different in C# compared to C++: In C#, you are allowed to use a string as the variable being tested. Loops C# provides four different loops (for, while, do. . .while, and foreach) that enable you to execute a block of code repeatedly until a certain condition is met. The for Loop C# for loops provide a mechanism for iterating through a loop whereby you test whether a particular condition holds true before you perform another iteration. The syntax is for (initializer; condition; iterator): statement(s) where: ➤➤ The initializer is the expression evaluated before the first loop is executed (usually initializing a local variable as a loop counter). ➤➤ The condition is the expression checked before each new iteration of the loop (this must evaluate to true for another iteration to be performed). ➤➤ The iterator is an expression evaluated after each iteration (usually incrementing the loop counter). The iterations end when the condition evaluates to false. The for loop is a so-called pretest loop because the loop condition is evaluated before the loop statements are executed; therefore, the contents of the loop won’t be executed at all if the loop condition is false. The for loop is excellent for repeating a statement or a block of statements for a predetermined number of times. The following example demonstrates typical usage of a for loop. It will write out all the integers from 0 to 99: for (int i = 0; i < 100; i=i+1) // This is equivalent to // For i = 0 To 99 in VB. { Console.WriteLine(i); } Here, you declare an int called i and initialize it to zero. This will be used as the loop counter. You then immediately test whether it is less than 100. Because this condition evaluates to true, you execute the code c02.indd 40 30-01-2014 20:04:52 Flow Control  ❘  41 in the loop, displaying the value 0. You then increment the counter by one, and walk through the process again. Looping ends when i reaches 100. Actually, the way the preceding loop is written isn’t quite how you would normally write it. C# has a shorthand for adding 1 to a variable, so instead of i = i + 1, you can simply write i++: for (int i = 0; i < 100; i++) { // etc. } You can also make use of type inference for the iteration variable i in the preceding example. Using type inference the loop construct would be as follows: for (var i = 0; i < 100; i++) .. It’s not unusual to nest for loops so that an inner loop executes once completely for each iteration of an outer loop. This approach is typically employed to loop through every element in a rectangular multidimensional array. The outermost loop loops through every row, and the inner loop loops through every column in a particular row. The following code displays rows of numbers. It also uses another Console method, Console. Write(), which does the same thing as Console.WriteLine() but doesn’t send a carriage return to the output: using System; namespace Wrox { class MainEntryPoint { static void Main(string[] args) { // This loop iterates through rows for (int i = 0; i < 100; i+=10) { // This loop iterates through columns for (int j = i; j < i + 10; j++) { Console.Write(" " + j); } Console.WriteLine(); } } } } Although j is an integer, it is automatically converted to a string so that the concatenation can take place. The preceding sample results in this output: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 It is technically possible to evaluate something other than a counter variable in a for loop’s test condition, but it is certainly not typical. It is also possible to omit one (or even all) of the expressions in the for loop. In such situations, however, you should consider using the while loop. c02.indd 41 30-01-2014 20:04:52 42  ❘  CHAPTER 2  Core C# The while Loop Like the for loop, while is a pretest loop. The syntax is similar, but while loops take only one expression: while(condition) statement(s); Unlike the for loop, the while loop is most often used to repeat a statement or a block of statements for a number of times that is not known before the loop begins. Usually, a statement inside the while loop’s body will set a Boolean flag to false on a certain iteration, triggering the end of the loop, as in the following example: bool condition = false; while (!condition) { // This loop spins until the condition is true. DoSomeWork(); condition = CheckCondition(); // assume CheckCondition() returns a bool } The do. . .while Loop The do...while loop is the post-test version of the while loop. This means that the loop’s test condition is evaluated after the body of the loop has been executed. Consequently, do...while loops are useful for situations in which a block of statements must be executed at least one time, as in this example: bool condition; do { // This loop will at least execute once, even if Condition is false. MustBeCalledAtLeastOnce(); condition = CheckCondition(); } while (condition); The foreach Loop The foreach loop enables you to iterate through each item in a collection. For now, don’t worry about exactly what a collection is (it is explained fully in Chapter 10, “Collections”); just understand that it is an object that represents a list of objects. Technically, to count as a collection, it must support an interface called IEnumerable. Examples of collections include C# arrays, the collection classes in the System. Collection namespaces, and user-defined collection classes. You can get an idea of the syntax of foreach from the following code, if you assume that arrayOfInts is (unsurprisingly) an array of ints: foreach (int temp in arrayOfInts) { Console.WriteLine(temp); } Here, foreach steps through the array one element at a time. With each element, it places the value of the element in the int variable called temp and then performs an iteration of the loop. Here is another situation where type inference can be used. The foreach loop would become the following: foreach (var temp in arrayOfInts) .. temp would be inferred to int because that is what the collection item type is. An important point to note with foreach is that you can’t change the value of the item in the collection (temp in the preceding code), so code such as the following will not compile: foreach (int temp in arrayOfInts) { temp++; Console.WriteLine(temp); } c02.indd 42 30-01-2014 20:04:52 Enumerations  ❘  43 If you need to iterate through the items in a collection and change their values, you must use a for loop instead. Jump Statements C# provides a number of statements that enable you to jump immediately to another line in the program. The first of these is, of course, the notorious goto statement. The goto Statement The goto statement enables you to jump directly to another specified line in the program, indicated by a label (this is just an identifier followed by a colon): goto Label1; Console.WriteLine("This won't be executed"); Label1: Console.WriteLine("Continuing execution from here"); A couple of restrictions are involved with goto. You can’t jump into a block of code such as a for loop, you can’t jump out of a class, and you can’t exit a finally block after try.catch blocks (Chapter 16, “Errors and Exceptions,” looks at exception handling with try.catch.finally). The reputation of the goto statement probably precedes it, and in most circumstances, its use is sternly frowned upon. In general, it certainly doesn’t conform to good object-oriented programming practices. The break Statement You have already met the break statement briefly — when you used it to exit from a case in a switch statement. In fact, break can also be used to exit from for, foreach, while, or do..while loops. Control will switch to the statement immediately after the end of the loop. If the statement occurs in a nested loop, control switches to the end of the innermost loop. If the break occurs outside of a switch statement or a loop, a compile-time error will occur. The continue Statement The continue statement is similar to break, and must also be used within a for, foreach, while, or do.. while loop. However, it exits only from the current iteration of the loop, meaning that execution will restart at the beginning of the next iteration of the loop, rather than outside the loop altogether. The return Statement The return statement is used to exit a method of a class, returning control to the caller of the method. If the method has a return type, return must return a value of this type; otherwise, if the method returns void, you should use return without an expression. Enumerations An enumeration is a user-defined integer type. When you declare an enumeration, you specify a set of acceptable values that instances of that enumeration can contain. Not only that, but you can also give the values user-friendly names. If, somewhere in your code, you attempt to assign a value that is not in the acceptable set of values to an instance of that enumeration, the compiler will flag an error. Creating an enumeration can save you a lot of time and headaches in the long run. At least three benefits exist to using enumerations instead of plain integers: ➤➤ As mentioned, enumerations make your code easier to maintain by helping to ensure that your variables are assigned only legitimate, anticipated values. c02.indd 43 30-01-2014 20:04:52 44  ❘  CHAPTER 2  Core C# ➤➤ Enumerations make your code clearer by allowing you to refer to integer values by descriptive names rather than by obscure “magic” numbers. ➤➤ Enumerations make your code easier to type, too. When you begin to assign a value to an instance of an enumerated type, the Visual Studio .NET IDE will, through IntelliSense, pop up a list box of acceptable values to save you some keystrokes and remind you of the possible options. You can define an enumeration as follows: public enum TimeOfDay { Morning = 0, Afternoon = 1, Evening = 2 } In this case, you use an integer value to represent each period of the day in the enumeration. You can now access these values as members of the enumeration. For example, TimeOfDay.Morning will return the value 0. You will typically use this enumeration to pass an appropriate value into a method and iterate through the possible values in a switch statement: class EnumExample { public static int Main() { WriteGreeting(TimeOfDay.Morning); return 0; } static void WriteGreeting(TimeOfDay timeOfDay) { switch(timeOfDay) { case TimeOfDay.Morning: Console.WriteLine("Good morning!"); break; case TimeOfDay.Afternoon: Console.WriteLine("Good afternoon!"); break; case TimeOfDay.Evening: Console.WriteLine("Good evening!"); break; default: Console.WriteLine("Hello!"); break; } } } The real power of enums in C# is that behind the scenes they are instantiated as structs derived from the base class, System.Enum. This means it is possible to call methods against them to perform some useful tasks. Note that because of the way the .NET Framework is implemented, no performance loss is associated with treating the enums syntactically as structs. In practice, after your code is compiled, enums will exist as primitive types, just like int and float. You can retrieve the string representation of an enum, as in the following example, using the earlier TimeOfDay enum: TimeOfDay time = TimeOfDay.Afternoon; Console.WriteLine(time.ToString()); This returns the string Afternoon. Alternatively, you can obtain an enum value from a string: TimeOfDay time2 = (TimeOfDay) Enum.Parse(typeof(TimeOfDay), "afternoon", true); Console.WriteLine((int)time2); c02.indd 44 30-01-2014 20:04:52 Namespaces  ❘  45 This code snippet illustrates both obtaining an enum value from a string and converting to an integer. To convert from a string, you need to use the static Enum.Parse() method, which, as shown, takes three parameters. The first is the type of enum you want to consider. The syntax is the keyword typeof followed by the name of the enum class in brackets. (Chapter 7 explores the typeof operator in more detail.) The second parameter is the string to be converted, and the third parameter is a bool indicating whether case should be ignored when doing the conversion. Finally, note that Enum.Parse() actually returns an object reference — you need to explicitly convert this to the required enum type (this is an example of an unboxing operation). For the preceding code, this returns the value 1 as an object, corresponding to the enum value of TimeOfDay.Afternoon. Converting explicitly to an int, this produces the value 1 again. Other methods on System.Enum do things such as return the number of values in an enum definition or list the names of the values. Full details are in the MSDN documentation. Namespaces As you saw earlier in this chapter, namespaces provide a way to organize related classes and other types. Unlike a file or a component, a namespace is a logical, rather than a physical, grouping. When you define a class in a C# file, you can include it within a namespace definition. Later, when you define another class that performs related work in another file, you can include it within the same namespace, creating a logical grouping that indicates to other developers using the classes how they are related and used: namespace CustomerPhoneBookApp { using System; public struct Subscriber { // Code for struct here.. } } Placing a type in a namespace effectively gives that type a long name, consisting of the type’s namespace as a series of names separated with periods (.), terminating with the name of the class. In the preceding example, the full name of the Subscriber struct is CustomerPhoneBookApp.Subscriber. This enables distinct classes with the same short name to be used within the same program without ambiguity. This full name is often called the fully qualified name. You can also nest namespaces within other namespaces, creating a hierarchical structure for your types: namespace Wrox { namespace ProCSharp { namespace Basics { class NamespaceExample { // Code for the class here.. } } } } Each namespace name is composed of the names of the namespaces it resides within, separated with periods, starting with the outermost namespace and ending with its own short name. Therefore, the full name for the ProCSharp namespace is Wrox.ProCSharp, and the full name of the NamespaceExample class is Wrox. ProCSharp.Basics.NamespaceExample. You can use this syntax to organize the namespaces in your namespace definitions too, so the previous code could also be written as follows: namespace Wrox.ProCSharp.Basics { c02.indd 45 30-01-2014 20:04:52 46  ❘  CHAPTER 2  Core C# class NamespaceExample { // Code for the class here.. } } Note that you are not permitted to declare a multipart namespace nested within another namespace. Namespaces are not related to assemblies. It is perfectly acceptable to have different namespaces in the same assembly or to define types in the same namespace in different assemblies. Defining the namespace hierarchy should be planned out prior to the start of a project. Generally the accepted format is CompanyName.ProjectName.SystemSection. In the previous example, Wrox is the company name, ProCSharp is the project, and in the case of this chapter, Basics is the section. The using Directive Obviously, namespaces can grow rather long and tiresome to type, and the capability to indicate a particular class with such specificity may not always be necessary. Fortunately, as noted earlier in this chapter, C# allows you to abbreviate a class’s full name. To do this, list the class’s namespace at the top of the file, prefixed with the using keyword. Throughout the rest of the file, you can refer to the types in the namespace simply by their type names: using System; using Wrox.ProCSharp; As remarked earlier, virtually all C# source code will have the statement using System; simply because so many useful classes supplied by Microsoft are contained in the System namespace. If two namespaces referenced by using statements contain a type of the same name, you need to use the full (or at least a longer) form of the name to ensure that the compiler knows which type to access. For example, suppose classes called NamespaceExample exist in both the Wrox.ProCSharp.Basics and Wrox .ProCSharp.OOP namespaces. If you then create a class called Test in the Wrox.ProCSharp namespace, and instantiate one of the NamespaceExample classes in this class, you need to specify which of these two classes you’re talking about: using Wrox.ProCSharp.OOP; using Wrox.ProCSharp.Basics; namespace Wrox.ProCSharp { class Test { public static int Main() { Basics.NamespaceExample nSEx = new Basics.NamespaceExample(); // do something with the nSEx variable. return 0; } } | Note  Because using statements occur at the top of C# files, in the same place that C and C++ list #include statements, it’s easy for programmers moving from C++ to C# to confuse namespaces with C++-style header files. Don’t make this mistake. The using statement does no physical linking between files, and C# has no equivalent to C++ header files. Your organization will probably want to spend some time developing a namespace convention so that its developers can quickly locate functionality that they need and so that the names of the organization’s c02.indd 46 30-01-2014 20:04:53 The Main() Method  ❘  47 homegrown classes won’t conflict with those in off-the-shelf class libraries. Guidelines on establishing your own namespace convention, along with other naming recommendations, are discussed later in this chapter. Namespace Aliases Another use of the using keyword is to assign aliases to classes and namespaces. If you need to refer to a very long namespace name several times in your code but don’t want to include it in a simple using statement (for example, to avoid type name conflicts), you can assign an alias to the namespace. The syntax for this is as follows: using alias = NamespaceName; The following example (a modified version of the previous example) assigns the alias Introduction to the Wrox.ProCSharp.Basics namespace and uses this to instantiate a NamespaceExample object, which is defined in this namespace. Notice the use of the namespace alias qualifier (::). This forces the search to start with the Introduction namespace alias. If a class called Introduction had been introduced in the same scope, a conflict would occur. The :: operator enables the alias to be referenced even if the conflict exists. The NamespaceExample class has one method, GetNamespace(), which uses the GetType() method exposed by every class to access a Type object representing the class’s type. You use this object to return a name of the class’s namespace: using System; using Introduction = Wrox.ProCSharp.Basics; class Test { public static int Main() { Introduction::NamespaceExample NSEx = new Introduction::NamespaceExample(); Console.WriteLine(NSEx.GetNamespace()); return 0; } } namespace Wrox.ProCSharp.Basics { class NamespaceExample { public string GetNamespace() { return this.GetType().Namespace; } } } The Main() Method As described at the beginning of this chapter, C# programs start execution at a method named Main(). This must be a static method of a class (or struct), and must have a return type of either int or void. Although it is common to specify the public modifier explicitly, because by definition the method must be called from outside the program, it doesn’t actually matter what accessibility level you assign to the entry- point method — it will run even if you mark the method as private. Multiple Main() Methods When a C# console or Windows application is compiled, by default the compiler looks for exactly one Main() method in any class matching the signature that was just described and makes that class method c02.indd 47 30-01-2014 20:04:53 48  ❘  CHAPTER 2  Core C# the entry point for the program. If there is more than one Main() method, the compiler returns an error message. For example, consider the following code called DoubleMain.cs: using System; namespace Wrox { class Client { public static int Main() { MathExample.Main(); return 0; } } class MathExample { static int Add(int x, int y) { return x + y; } public static int Main() { int i = Add(5,10); Console.WriteLine(i); return 0; } } } This contains two classes, both of which have a Main() method. If you try to compile this code in the usual way, you will get the following errors: csc DoubleMain.cs Microsoft (R) Visual C# 2010 Compiler version 4.0.20506.1 Copyright (C) Microsoft Corporation. All rights reserved. DoubleMain.cs(7,25): error CS0017: Program 'DoubleMain.exe' has more than one entry point defined: 'Wrox.Client.Main()'. Compile with /main to specify the type that contains the entry point. DoubleMain.cs(21,25): error CS0017: Program 'DoubleMain.exe' has more than one entry point defined: 'Wrox.MathExample.Main()'. Compile with /main to specify the type that contains the entry point. However, you can explicitly tell the compiler which of these methods to use as the entry point for the program by using the /main switch, together with the full name (including namespace) of the class to which the Main() method belongs: csc DoubleMain.cs /main:Wrox.MathExample Passing Arguments to Main() The examples so far have shown only the Main() method without any parameters. However, when the program is invoked, you can get the CLR to pass any command-line arguments to the program by including a parameter. This parameter is a string array, traditionally called args (although C# will accept any name). The program can use this array to access any options passed through the command line when the program is started. The following example, ArgsExample.cs, loops through the string array passed in to the Main() method and writes the value of each option to the console window: using System; namespace Wrox { c02.indd 48 30-01-2014 20:04:53 More on Compiling C# Files  ❘  49 class ArgsExample { public static int Main(string[] args) { for (int i = 0; i < args.Length; i++) { Console.WriteLine(args[i]); } return 0; } } } You can compile this as usual using the command line. When you run the compiled executable, you can pass in arguments after the name of the program, as shown here: ArgsExample /a /b /c /a /b /c More on Compiling C# Files You have seen how to compile console applications using csc.exe, but what about other types of applications? What if you want to reference a class library? The full set of compilation options for the C# compiler is, of course, detailed in the MSDN documentation, but we list here the most important options. To answer the first question, you can specify what type of file you want to create using the /target switch, often abbreviated as /t. This can be one of those shown in the following table. Option Output /t:exe A console application (the default) /t:library A class library with a manifest /t:module A component without a manifest /t:winexe A Windows application (without a console window) If you want a nonexecutable file (such as a DLL) to be loadable by the .NET runtime, you must compile it as a library. If you compile a C# file as a module, no assembly will be created. Although modules cannot be loaded by the runtime, they can be compiled into another manifest using the /addmodule switch. Another option to be aware of is /out. This enables you to specify the name of the output file produced by the compiler. If the /out option isn’t specified, the compiler bases the name of the output file on the name of the input C# file, adding an extension according to the target type (for example, exe for a Windows or console application, or dll for a class library). Note that the /out and /t, or /target, options must precede the name of the file you want to compile. If you want to reference types in assemblies that aren’t referenced by default, you can use the /reference or /r switch, together with the path and filename of the assembly. The following example demonstrates how you can compile a class library and then reference that library in another assembly. It consists of two files: ➤➤ The class library ➤➤ A console application, which will call a class in the library The first file is called MathLibrary.cs and contains the code for your DLL. To keep things simple, it contains just one (public) class, MathLib, with a single method that adds two ints: namespace Wrox { public class MathLib { c02.indd 49 30-01-2014 20:04:53 50  ❘  CHAPTER 2  Core C# public int Add(int x, int y) { return x + y; } } } You can compile this C# file into a .NET DLL using the following command: csc /t:library MathLibrary.cs The console application, MathClient.cs, will simply instantiate this object and call its Add() method, displaying the result in the console window: using System; namespace Wrox { class Client { public static void Main() { MathLib mathObj = new MathLib(); Console.WriteLine(mathObj.Add(7,8)); } } } To compile this code, use the /r switch to point at or reference the newly compiled DLL: csc MathClient.cs /r:MathLibrary.dll You can then run it as normal just by entering MathClient at the command prompt. This displays the number 15 — the result of your addition. Console I/O By this point, you should have a basic familiarity with C#’s data types, as well as some knowledge of how the thread-of-control moves through a program that manipulates those data types. In this chapter, you have also used several of the Console class’s static methods used for reading and writing data. Because these methods are so useful when writing basic C# programs, this section briefly reviews them in more detail. To read a line of text from the console window, you use the Console.ReadLine() method. This reads an input stream (terminated when the user presses the Return key) from the console window and returns the input string. There are also two corresponding methods for writing to the console, which you have already used extensively: ➤➤ Console.Write() — Writes the specified value to the console window. ➤➤ Console.WriteLine() — Writes the specified value to the console window but adds a newline character at the end of the output. Various forms (overloads) of these methods exist for all the predefined types (including object), so in most cases you don’t have to convert values to strings before you display them. For example, the following code lets the user input a line of text and then displays that text: string s = Console.ReadLine(); Console.WriteLine(s); Console.WriteLine() also allows you to display formatted output in a way comparable to C’s printf() function. To use WriteLine() in this way, you pass in a number of parameters. The first is a string containing markers in curly braces where the subsequent parameters will be inserted into the text. Each c02.indd 50 30-01-2014 20:04:53 Console I/O  ❘  51 marker contains a zero-based index for the number of the parameter in the following list. For example, {0} represents the first parameter in the list. Consider the following code: int i = 10; int j = 20; Console.WriteLine("{0} plus {1} equals {2}", i, j, i + j); The preceding code displays the following: 10 plus 20 equals 30 You can also specify a width for the value, and justify the text within that width, using positive values for right justification and negative values for left justification. To do this, use the format {n,w}, where n is the parameter index and w is the width value: int i = 940; int j = 73; Console.WriteLine(" {0,4}\n+{1,4}\n —  — \n {2,4}", i, j, i + j); The result of the preceding is as follows: 940 + 73  —  —  1013 Finally, you can also add a format string, together with an optional precision value. It is not possible to provide a complete list of potential format strings because, as you will see in Chapter 9, “Strings and Regular Expressions,” you can define your own format strings. However, the main ones in use for the predefined types are described in the following table. String Description C Local currency format D Decimal format. Converts an integer to base 10, and pads with leading zeros if a precision specifier is given. E Scientific (exponential) format. The precision specifier sets the number of decimal places (6 by default). The case of the format string (e or E) determines the case of the exponential symbol. F Fixed-point format; the precision specifier controls the number of decimal places. Zero is acceptable. G General format. Uses E or F formatting, depending on which is more compact. N Number format. Formats the number with commas as the thousands separators — for example 32,767.44. P Percent format X Hexadecimal format. The precision specifier can be used to pad with leading zeros. Note that the format strings are normally case insensitive, except for e/E. If you want to use a format string, you should place it immediately after the marker that specifies the parameter number and field width, and separate it with a colon. For example, to format a decimal value as currency for the computer’s locale, with precision to two decimal places, you would use C2: decimal i = 940.23m; decimal j = 73.7m; Console.WriteLine(" {0,9:C2}\n+{1,9:C2}\n —  —  —  — -\n {2,9:C2}", i, j, i + j); The output of this in U.S. currency is as follows: $940.23 + $73.70  —  —  —  — - $1,013.93 c02.indd 51 30-01-2014 20:04:53 52  ❘  CHAPTER 2  Core C# As a final trick, you can also use placeholder characters instead of these format strings to map out formatting, as shown in this example: double d = 0.234; Console.WriteLine("{0:#.00}", d); This displays as .23 because the # symbol is ignored if there is no character in that place, and zeros are either replaced by the character in that position if there is one or printed as a zero. Using Comments The next topic — adding comments to your code — looks very simple on the surface, but can be complex. Comments can be beneficial to the other developers that may look at your code. Also, as you will see, they can be used to generate documentation of your code for developers to use. Internal Comments within the Source Files As noted earlier in this chapter, C# uses the traditional C-type single-line (//..) and multiline (/* .. */) comments: // This is a single-line comment /* This comment spans multiple lines. */ Everything in a single-line comment, from the // to the end of the line, is ignored by the compiler, and everything from an opening /* to the next */ in a multiline comment combination is ignored. Obviously, you can’t include the combination */ in any multiline comments, because this will be treated as the end of the comment. It is possible to put multiline comments within a line of code: Console.WriteLine(/* Here's a comment! */ "This will compile."); Use inline comments with care because they can make code hard to read. However, they can be useful when debugging if, for example, you temporarily want to try running the code with a different value somewhere: DoSomething(Width, /*Height*/ 100); Comment characters included in string literals are, of course, treated like normal characters: string s = "/* This is just a normal string .*/"; XML Documentation In addition to the C-type comments, illustrated in the preceding section, C# has a very neat feature that we want to highlight: the capability to produce documentation in XML format automatically from special comments. These comments are single-line comments but begin with three slashes (///) instead of the usual two. Within these comments, you can place XML tags containing documentation of the types and type members in your code. The tags in the following table are recognized by the compiler. Tag Description Marks up text within a line as code — for example, int i = 10;. Marks multiple lines as code Marks up a code example Documents an exception class. (Syntax is verified by the compiler.) Includes comments from another documentation file. (Syntax is verified by the compiler.) Inserts a list into the documentation c02.indd 52 30-01-2014 20:04:54 Using Comments  ❘  53 Tag Description Gives structure to text Marks up a method parameter. (Syntax is verified by the compiler.) Indicates that a word is a method parameter. (Syntax is verified by the compiler.) Documents access to a member. (Syntax is verified by the compiler.) Adds a description for a member Documents the return value for a method Provides a cross-reference to another parameter. (Syntax is verified by the compiler.) Provides a “see also” section in a description. (Syntax is verified by the compiler.) Provides a short summary of a type or member Used in the comment of a generic type to describe a type parameter The name of the type parameter Describes a property To see how this works, add some XML comments to the MathLibrary.cs file from the previous “More on Compiling C# Files” section. You will add a element for the class and for its Add() method, and a element and two elements for the Add() method: // MathLib.cs namespace Wrox { /// /// Wrox.Math class. /// Provides a method to add two integers. /// public class MathLib { /// /// The Add method allows us to add two integers. /// ///Result of the addition (int) ///First number to add ///Second number to add public int Add(int x, int y) { return x + y; } } } The C# compiler can extract the XML elements from the special comments and use them to generate an XML file. To get the compiler to generate the XML documentation for an assembly, you specify the /doc option when you compile, together with the name of the file you want to be created: csc /t:library /doc:MathLibrary.xml MathLibrary.cs The compiler will throw an error if the XML comments do not result in a well-formed XML document. The preceding will generate an XML file named Math.xml, which looks like this: MathLibrary c02.indd 53 30-01-2014 20:04:54 54  ❘  CHAPTER 2  Core C# Wrox.MathLibrary class. Provides a method to add two integers. The Add method allows us to add two integers. Result of the addition (int) First number to add Second number to add Notice how the compiler has actually done some work for you; it has created an element and added a element for each type or member of a type in the file. Each element has a name attribute with the full name of the member as its value, prefixed by a letter that indicates whether it is a type (T:), field (F:), or member (M:). The C# Preprocessor Directives Besides the usual keywords, most of which you have now encountered, C# also includes a number of commands that are known as preprocessor directives. These commands are never actually translated to any commands in your executable code, but they affect aspects of the compilation process. For example, you can use preprocessor directives to prevent the compiler from compiling certain portions of your code. You might do this if you are planning to release two versions of it — a basic version and an enterprise version that will have more features. You could use preprocessor directives to prevent the compiler from compiling code related to the additional features when you are compiling the basic version of the software. In another scenario, you might have written bits of code that are intended to provide you with debugging information. You probably don’t want those portions of code compiled when you actually ship the software. The preprocessor directives are all distinguished by beginning with the # symbol. Note  C++ developers will recognize the preprocessor directives as something that plays an important part in C and C++. However, there aren’t as many preprocessor directives in C#, and they are not used as often. C# provides other mechanisms, such as custom attributes, that achieve some of the same effects as C++ directives. Also, note that C# doesn’t actually have a separate preprocessor in the way that C++ does. The so-called preprocessor directives are actually handled by the compiler. Nevertheless, C# retains the name preprocessor directive because these commands give the impression of a preprocessor. The following sections briefly cover the purposes of the preprocessor directives. #define and #undef #define is used like this: #define DEBUG This tells the compiler that a symbol with the given name (in this case DEBUG) exists. It is a little bit like declaring a variable, except that this variable doesn’t really have a value — it just exists. Also, this symbol isn’t part of your actual code; it exists only for the benefit of the compiler, while the compiler is compiling the code, and has no meaning within the C# code itself. c02.indd 54 30-01-2014 20:04:54 The C# Preprocessor Directives  ❘  55 #undef does the opposite, and removes the definition of a symbol: #undef DEBUG If the symbol doesn’t exist in the first place, then #undef has no effect. Similarly, #define has no effect if a symbol already exists. You need to place any #define and #undef directives at the beginning of the C# source file, before any code that declares any objects to be compiled. #define isn’t much use on its own, but when combined with other preprocessor directives, especially #if, it becomes very powerful. Note  Incidentally, you might notice some changes from the usual C# syntax. Preprocessor directives are not terminated by semicolons and they normally constitute the only command on a line. That’s because for the preprocessor directives, C# abandons its usual practice of requiring commands to be separated by semicolons. If the compiler sees a preprocessor directive, it assumes that the next command is on the next line. #if, #elif, #else, and #endif These directives inform the compiler whether to compile a block of code. Consider this method: int DoSomeWork(double x) { // do something #if DEBUG Console.WriteLine("x is " + x); #endif } This code will compile as normal except for the Console.WriteLine() method call contained inside the #if clause. This line will be executed only if the symbol DEBUG has been defined by a previous #define directive. When the compiler finds the #if directive, it checks to see whether the symbol concerned exists, and compiles the code inside the #if clause only if the symbol does exist. Otherwise, the compiler simply ignores all the code until it reaches the matching #endif directive. Typical practice is to define the symbol DEBUG while you are debugging and have various bits of debugging-related code inside #if clauses. Then, when you are close to shipping, you simply comment out the #define directive, and all the debugging code miraculously disappears, the size of the executable file gets smaller, and your end users don’t get confused by seeing debugging information. (Obviously, you would do more testing to ensure that your code still works without DEBUG defined.) This technique is very common in C and C++ programming and is known as conditional compilation. The #elif (=else if) and #else directives can be used in #if blocks and have intuitively obvious meanings. It is also possible to nest #if blocks: #define ENTERPRISE #define W2K // further on in the file #if ENTERPRISE // do something #if W2K // some code that is only relevant to enterprise // edition running on W2K #endif #elif PROFESSIONAL // do something else #else // code for the leaner version #endif c02.indd 55 30-01-2014 20:04:54 56  ❘  CHAPTER 2  Core C# Note  Unlike the situation in C++, using #if is not the only way to compile code conditionally. C# provides an alternative mechanism through the Conditional attribute, which is explored in Chapter 15, “Reflection.” #if and #elif support a limited range of logical operators too, using the operators !, ==, !=, and ||. A symbol is considered to be true if it exists and false if it doesn’t. For example: #if W2K && (ENTERPRISE==false) // if W2K is defined but ENTERPRISE isn't #warning and #error Two other very useful preprocessor directives are #warning and #error. These will respectively cause a warning or an error to be raised when the compiler encounters them. If the compiler sees a #warning directive, it displays whatever text appears after the #warning to the user, after which compilation continues. If it encounters a #error directive, it displays the subsequent text to the user as if it were a compilation error message and then immediately abandons the compilation, so no IL code will be generated. You can use these directives as checks that you haven’t done anything silly with your #define statements; you can also use the #warning statements to remind yourself to do something: #if DEBUG && RELEASE #error "You've defined DEBUG and RELEASE simultaneously!" #endif #warning "Don't forget to remove this line before the boss tests the code!" Console.WriteLine("*I hate this job.*"); #region and #endregion The #region and #endregion directives are used to indicate that a certain block of code is to be treated as a single block with a given name, like this: #region Member Field Declarations int x; double d; Currency balance; #endregion This doesn’t look that useful by itself; it doesn’t affect the compilation process in any way. However, the real advantage is that these directives are recognized by some editors, including the Visual Studio .NET editor. These editors can use the directives to lay out your code better on the screen. You will see how this works in Chapter 17. #line The #line directive can be used to alter the filename and line number information that is output by the compiler in warnings and error messages. You probably won’t want to use this directive very often. It’s most useful when you are coding in conjunction with another package that alters the code you are typing in before sending it to the compiler. In this situation, line numbers, or perhaps the filenames reported by the compiler, won’t match up to the line numbers in the files or the filenames you are editing. The #line directive can be used to restore the match. You can also use the syntax #line default to restore the line to the default line numbering: #line 164 "Core.cs" // We happen to know this is line 164 in the file // Core.cs, before the intermediate // package mangles it. // later on #line default // restores default line numbering c02.indd 56 30-01-2014 20:04:54 C# Programming Guidelines  ❘  57 #pragma The #pragma directive can either suppress or restore specific compiler warnings. Unlike command-line options, the #pragma directive can be implemented on the class or method level, enabling fine-grained control over what warnings are suppressed and when. The following example disables the “field not used” warning and then restores it after the MyClass class compiles: #pragma warning disable 169 public class MyClass { int neverUsedField; } #pragma warning restore 169 C# Programming Guidelines This final section of the chapter supplies the guidelines you need to bear in mind when writing C# programs. These are guidelines that most C# developers will use. By using these guidelines other developers will feel comfortable working with your code. Rules for Identifiers This section examines the rules governing what names you can use for variables, classes, methods, and so on. Note that the rules presented in this section are not merely guidelines: they are enforced by the C# compiler. Identifiers are the names you give to variables, to user-defined types such as classes and structs, and to members of these types. Identifiers are case sensitive, so, for example, variables named interestRate and InterestRate would be recognized as different variables. Following are a few rules determining what identifiers you can use in C#: ➤➤ They must begin with a letter or underscore, although they can contain numeric characters. ➤➤ You can’t use C# keywords as identifiers. The following table lists the C# reserved keywords. abstract event new struct as explicit null switch base extern object this bool false operator throw break finally out true byte fixed override try case float params typeof catch for private uint char foreach protected ulong checked goto public unchecked class if readonly unsafe const implicit ref ushort continue in return using decimal int sbyte virtual default interface sealed void (continues) c02.indd 57 30-01-2014 20:04:54 58  ❘  CHAPTER 2  Core C# abstract event new struct delegate internal short volatile do is sizeof while double lock stackalloc else long static enum namespace string If you need to use one of these words as an identifier (for example, if you are accessing a class written in a different language), you can prefix the identifier with the @ symbol to indicate to the compiler that what follows should be treated as an identifier, not as a C# keyword (so abstract is not a valid identifier, but @ abstract is). Finally, identifiers can also contain Unicode characters, specified using the syntax \uXXXX, where XXXX is the four-digit hex code for the Unicode character. The following are some examples of valid identifiers: ➤➤ Name ➤➤ Überfluß ➤➤ _Identifier ➤➤ \u005fIdentifier The last two items in this list are identical and interchangeable (because 005f is the Unicode code for the underscore character), so obviously these identifiers couldn’t both be declared in the same scope. Note that although syntactically you are allowed to use the underscore character in identifiers, this isn’t recommended in most situations. That’s because it doesn’t follow the guidelines for naming variables that Microsoft has written to ensure that developers use the same conventions, making it easier to read one another’s code. Usage Conventions In any development language, certain traditional programming styles usually arise. The styles are not part of the language itself but rather are conventions — for example, how variables are named or how certain classes, methods, or functions are used. If most developers using that language follow the same conventions, it makes it easier for different developers to understand each other’s code — which in turn generally helps program maintainability. Conventions do, however, depend on the language and the environment. For example, C++ developers programming on the Windows platform have traditionally used the prefixes psz or lpsz to indicate strings — char *pszResult; char *lpszMessage; — but on Unix machines it’s more common not to use any such prefixes: char *Result; char *Message;. You’ll notice from the sample code in this book that the convention in C# is to name variables without prefixes: string Result; string Message;. Note  The convention by which variable names are prefixed with letters that represent the data type is known as Hungarian notation. It means that other developers reading the code can immediately tell from the variable name what data type the variable represents. Hungarian notation is widely regarded as redundant in these days of smart editors and IntelliSense. Whereas with many languages usage conventions simply evolved as the language was used, with C# and the whole of the .NET Framework, Microsoft has written very comprehensive usage guidelines, which are detailed in the .NET/C# MSDN documentation. This means that, right from the start, .NET programs have a high degree of interoperability in terms of developers being able to understand code. The guidelines have (continued) c02.indd 58 30-01-2014 20:04:55 C# Programming Guidelines  ❘  59 also been developed with the benefit of some 20 years’ hindsight in object-oriented programming. Judging by the relevant newsgroups, the guidelines have been carefully thought out and are well received in the developer community. Hence, the guidelines are well worth following. Note, however, that the guidelines are not the same as language specifications. You should try to follow the guidelines when you can. Nevertheless, you won’t run into problems if you have a good reason for not doing so — for example, you won’t get a compilation error because you don’t follow these guidelines. The general rule is that if you don’t follow the usage guidelines, you must have a convincing reason. Departing from the guidelines should be a conscious decision rather than simply not bothering. Also, if you compare the guidelines with the samples in the remainder of this book, you’ll notice that in numerous examples we have chosen not to follow the conventions. That’s usually because the conventions are designed for much larger programs than our samples; and although they are great if you are writing a complete software package, they are not really suitable for small 20-line standalone programs. In many cases, following the conventions would have made our samples harder, rather than easier, to follow. The full guidelines for good programming style are quite extensive. This section is confined to describing some of the more important guidelines, as well as those most likely to surprise you. To be absolutely certain that your code follows the usage guidelines completely, you need to refer to the MSDN documentation. Naming Conventions One important aspect of making your programs understandable is how you choose to name your items — and that includes naming variables, methods, classes, enumerations, and namespaces. It is intuitively obvious that your names should reflect the purpose of the item and should not clash with other names. The general philosophy in the .NET Framework is also that the name of a variable should reflect the purpose of that variable instance and not the data type. For example, height is a good name for a variable, whereas integerValue isn’t. However, you are likely to find that principle an ideal that is hard to achieve. Particularly when you are dealing with controls, in most cases you’ll probably be happier sticking with variable names such as confirmationDialog and chooseEmployeeListBox, which do indicate the data type in the name. The following sections look at some of the things you need to think about when choosing names. Casing of Names In many cases you should use Pascal casing for names. With Pascal casing, the first letter of each word in a name is capitalized: EmployeeSalary, ConfirmationDialog, PlainTextEncoding. You will notice that nearly all the names of namespaces, classes, and members in the base classes follow Pascal casing. In particular, the convention of joining words using the underscore character is discouraged. Therefore, try not to use names such as employee_salary. It has also been common in other languages to use all capitals for names of constants. This is not advised in C# because such names are harder to read — the convention is to use Pascal casing throughout: const int MaximumLength; The only other casing convention that you are advised to use is camel casing. Camel casing is similar to Pascal casing, except that the first letter of the first word in the name is not capitalized: employeeSalary, confirmationDialog, plainTextEncoding. Following are three situations in which you are advised to use camel casing: ➤➤ For names of all private member fields in types: private int subscriberId; Note, however, that often it is conventional to prefix names of member fields with an underscore: private int _subscriberId; ➤➤ For names of all parameters passed to methods: public void RecordSale(string salesmanName, int quantity); c02.indd 59 30-01-2014 20:04:55 60  ❘  CHAPTER 2  Core C# ➤➤ To distinguish items that would otherwise have the same name. A common example is when a property wraps around a field: private string employeeName; public string EmployeeName { get { return employeeName; } } If you are doing this, you should always use camel casing for the private member and Pascal casing for the public or protected member, so that other classes that use your code see only names in Pascal case (except for parameter names). You should also be wary about case sensitivity. C# is case sensitive, so it is syntactically correct for names in C# to differ only by the case, as in the previous examples. However, bear in mind that your assemblies might at some point be called from Visual Basic .NET applications — and Visual Basic .NET is not case sensitive. Hence, if you do use names that differ only by case, it is important to do so only in situations in which both names will never be seen outside your assembly. (The previous example qualifies as okay because camel case is used with the name that is attached to a private variable.) Otherwise, you may prevent other code written in Visual Basic .NET from being able to use your assembly correctly. Name Styles Be consistent about your style of names. For example, if one of the methods in a class is called ShowConfirmationDialog(), then you should not give another method a name such as ShowDialogWarning() or WarningDialogShow(). The other method should be called ShowWarningDialog(). Namespace Names It is particularly important to choose Namespace names carefully to avoid the risk of ending up with the same name for one of your namespaces as someone else uses. Remember, namespace names are the only way that .NET distinguishes names of objects in shared assemblies. Therefore, if you use the same namespace name for your software package as another package, and both packages are installed on the same computer, problems will occur. Because of this, it’s almost always a good idea to create a top-level namespace with the name of your company and then nest successive namespaces that narrow down the technology, group, or department you are working in or the name of the package for which your classes are intended. Microsoft recommends namespace names that begin with ., as in these two examples: WeaponsOfDestructionCorp.RayGunControllers WeaponsOfDestructionCorp.Viruses Names and Keywords It is important that the names do not clash with any keywords. In fact, if you attempt to name an item in your code with a word that happens to be a C# keyword, you’ll almost certainly get a syntax error because the compiler will assume that the name refers to a statement. However, because of the possibility that your classes will be accessed by code written in other languages, it is also important that you don’t use names that are keywords in other .NET languages. Generally speaking, C++ keywords are similar to C# keywords, so confusion with C++ is unlikely, and those commonly encountered keywords that are unique to Visual C++ tend to start with two underscore characters. As with C#, C++ keywords are spelled in lowercase, so if you hold to the convention of naming your public classes and members with Pascal-style names, they will always have at least one uppercase letter in their names, and there will be no risk of clashes with C++ keywords. However, you are more likely to have problems with Visual Basic .NET, which has many more keywords than C# does, and being non-case-sensitive means that you cannot rely on Pascal-style names for your classes and methods. c02.indd 60 30-01-2014 20:04:55 C# Programming Guidelines  ❘  61 The following table lists the keywords and standard function calls in Visual Basic .NET, which you should avoid, if possible, in whatever case combination, for your public C# classes. Abs Do Loc RGB Add Double Local Right AddHandler Each Lock RmDir AddressOf Else LOF Rnd Alias ElseIf Log RTrim And Empty Long SaveSettings Ansi End Loop Second AppActivate Enum LTrim Seek Append EOF Me Select As Erase Mid SetAttr Asc Err Minute SetException Assembly Error MIRR Shared Atan Event MkDir Shell Auto Exit Module Short Beep Exp Month Sign Binary Explicit MustInherit Sin BitAnd ExternalSource MustOverride Single BitNot False MyBase SLN BitOr FileAttr MyClass Space BitXor FileCopy Namespace Spc Boolean FileDateTime New Split ByRef FileLen Next Sqrt Byte Filter Not Static ByVal Finally Nothing Step Call Fix NotInheritable Stop Case For NotOverridable Str Catch Format Now StrComp CBool FreeFile NPer StrConv CByte Friend NPV Strict CDate Function Null String CDbl FV Object Structure CDec Get Oct Sub ChDir GetAllSettings Off Switch ChDrive GetAttr On SYD Choose GetException Open SyncLock Chr GetObject Option Tab CInt GetSetting Optional Tan Class GetType Or Text Clear GoTo Overloads Then CLng Handles Overridable Throw (continues) c02.indd 61 30-01-2014 20:04:55 62  ❘  CHAPTER 2  Core C# Abs Do Loc RGB Close Hex Overrides TimeOfDay Collection Hour ParamArray Timer Command If Pmt TimeSerial Compare Iif PPmt TimeValue Const Implements Preserve To Cos Imports Print Today CreateObject In Private Trim CShort Inherits Property Try CSng Input Public TypeName CStr InStr Put TypeOf CurDir Int PV UBound Date Integer QBColor UCase DateAdd Interface Raise Unicode DateDiff Ipmt RaiseEvent Unlock DatePart IRR Randomize Until DateSerial Is Rate Val DateValue IsArray Read Weekday Day IsDate ReadOnly While DDB IsDbNull ReDim Width Decimal IsNumeric Remove With Declare Item RemoveHandler WithEvents Default Kill Rename Write Delegate Lcase Replace WriteOnly DeleteSetting Left Reset Xor Dim Lib Resume Year Use of Properties and Methods One area that can cause confusion regarding a class is whether a particular quantity should be represented by a property or a method. The rules are not hard and fast, but in general you should use a property if something should look and behave like a variable. (If you’re not sure what a property is, see Chapter 3.) This means, among other things, that: ➤➤ Client code should be able to read its value. Write-only properties are not recommended, so, for example, use a SetPassword() method, not a write-only Password property. ➤➤ Reading the value should not take too long. The fact that something is a property usually suggests that reading it will be relatively quick. ➤➤ Reading the value should not have any observable and unexpected side effect. Furthermore, setting the value of a property should not have any side effect that is not directly related to the property. Setting the width of a dialog has the obvious effect of changing the appearance of the dialog on the screen. That’s fine, because that’s obviously related to the property in question. ➤➤ It should be possible to set properties in any order. In particular, it is not good practice when setting a property to throw an exception because another related property has not yet been set. For example, to use a class that accesses a database, you need to set ConnectionString, UserName, and Password, and then the author of the class should ensure that the class is implemented such that users can set them in any order. (continued) c02.indd 62 30-01-2014 20:04:56 Summary  ❘  63 ➤➤ Successive reads of a property should give the same result. If the value of a property is likely to change unpredictably, you should code it as a method instead. Speed, in a class that monitors the motion of an automobile, is not a good candidate for a property. Use a GetSpeed() method here; but, Weight and EngineSize are good candidates for properties because they will not change for a given object. If the item you are coding satisfies all the preceding criteria, it is probably a good candidate for a property. Otherwise, you should use a method. Use of Fields The guidelines are pretty simple here. Fields should almost always be private, although in some cases it may be acceptable for constant or read-only fields to be public. Making a field public may hinder your ability to extend or modify the class in the future. The previous guidelines should give you a foundation of good practices, and you should use them in conjunction with a good object-oriented programming style. A final helpful note to keep in mind is that Microsoft has been relatively careful about being consistent and has followed its own guidelines when writing the .NET base classes, so a very good way to get an intuitive feel for the conventions to follow when writing .NET code is to simply look at the base classes — see how classes, members, and namespaces are named, and how the class hierarchy works. Consistency between the base classes and your classes will facilitate readability and maintainability. Summary This chapter examined some of the basic syntax of C#, covering the areas needed to write simple C# programs. We covered a lot of ground, but much of it will be instantly recognizable to developers who are familiar with any C-style language (or even JavaScript). You have seen that although C# syntax is similar to C++ and Java syntax, there are many minor differences. You have also seen that in many areas this syntax is combined with facilities to write code very quickly — for example, high-quality string handling facilities. C# also has a strongly defined type system, based on a distinction between value and reference types. Chapters 3 and 4, “Objects and Types” and “Inheritance” respectively, cover the C# object-oriented programming features. c02.indd 63 30-01-2014 20:04:56 c02.indd 64 30-01-2014 20:04:56 Objects and Types WHAT’s IN THIs CHAPTER? ➤➤ The differences between classes and structs ➤➤ Class members ➤➤ Passing values by value and by reference ➤➤ Method overloading ➤➤ Constructors and static constructors ➤➤ R e a d - o n l y fi e l d s ➤➤ Partial classes ➤➤ Static classes ➤➤ Weak references ➤➤ The Object class, from which all other types are derived WRox.CoM CodE doWNloAds FoR THIs CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ MathTest ➤➤ MathTestWeakReference ➤➤ ParameterTest CREATING ANd usING ClAssEs So far, you’ve been introduced to some of the building blocks of the C# language, including variables, data types, and program fl ow statements, and you have seen a few very short complete programs containing little more than the Main() method. What you haven’t seen yet is how to put all these elements together to form a longer, complete program. The key to this lies in working with classes — the subject of this chapter. Note that we cover inheritance and features related to inheritance in Chapter 4, “Inheritance.” 3 c03.indd 65 30-01-2014 20:05:38 66  ❘  CHAPTER 3  Objects and Types Note  This chapter introduces the basic syntax associated with classes. However, we assume that you are already familiar with the underlying principles of using classes — for example, that you know what a constructor or a property is. This chapter is largely confined to applying those principles in C# code. Classes and Structs Classes and structs are essentially templates from which you can create objects. Each object contains data and has methods to manipulate and access that data. The class defines what data and behavior each particular object (called an instance) of that class can contain. For example, if you have a class that represents a customer, it might define fields such as CustomerID, FirstName, LastName, and Address, which are used to hold information about a particular customer. It might also define functionality that acts upon the data stored in these fields. You can then instantiate an object of this class to represent one specific customer, set the field values for that instance, and use its functionality: class PhoneCustomer { public const string DayOfSendingBill = "Monday"; public int CustomerID; public string FirstName; public string LastName; } Structs differ from classes in the way that they are stored in memory and accessed (classes are reference types stored in the heap; structs are value types stored on the stack), and in some of their features (for example, structs don’t support inheritance). You typically use structs for smaller data types for performance reasons. In terms of syntax, however, structs look very similar to classes; the main difference is that you use the keyword struct instead of class to declare them. For example, if you wanted all PhoneCustomer instances to be allocated on the stack instead of the managed heap, you could write the following: struct PhoneCustomerStruct { public const string DayOfSendingBill = "Monday"; public int CustomerID; public string FirstName; public string LastName; } For both classes and structs, you use the keyword new to declare an instance. This keyword creates the object and initializes it; in the following example, the default behavior is to zero out its fields: PhoneCustomer myCustomer = new PhoneCustomer(); // works for a class PhoneCustomerStruct myCustomer2 = new PhoneCustomerStruct();// works for a struct In most cases, you’ll use classes much more often than structs. Therefore, we discuss classes first and then the differences between classes and structs and the specific reasons why you might choose to use a struct instead of a class. Unless otherwise stated, however, you can assume that code presented for a class will work equally well for a struct. Classes The data and functions within a class are known as the class’s members. Microsoft’s official terminology distinguishes between data members and function members. In addition to these members, classes can contain nested types (such as other classes). Accessibility to the members can be public, protected, internal protected, private, or internal. These are described in detail in Chapter 5, “Generics.” c03.indd 66 30-01-2014 20:05:38 Classes  ❘  67 Data Members Data members are those members that contain the data for the class — fields, constants, and events. Data members can be static. A class member is always an instance member unless it is explicitly declared as static. Fields are any variables associated with the class. You have already seen fields in use in the PhoneCustomer class in the previous example. After you have instantiated a PhoneCustomer object, you can then access these fields using the Object .FieldName syntax, as shown in this example: PhoneCustomer Customer1 = new PhoneCustomer(); Customer1.FirstName = "Simon"; Constants can be associated with classes in the same way as variables. You declare a constant using the const keyword. If it is declared as public, then it will be accessible from outside the class: class PhoneCustomer { public const string DayOfSendingBill = "Monday"; public int CustomerID; public string FirstName; public string LastName; } Events are class members that allow an object to notify a subscriber whenever something noteworthy happens, such as a field or property of the class changing, or some form of user interaction occurring. The client can have code, known as an event handler, that reacts to the event. Chapter 8, “Delegates, Lambdas, and Events,” looks at events in detail. Function Members Function members are those members that provide some functionality for manipulating the data in the class. They include methods, properties, constructors, finalizers, operators, and indexers. ➤➤ Methods are functions associated with a particular class. Like data members, function members are instance members by default. They can be made static by using the static modifier. ➤➤ Properties are sets of functions that can be accessed from the client in a similar way to the public fields of the class. C# provides a specific syntax for implementing read and write properties on your classes, so you don’t have to use method names that have the words Get or Set embedded in them. Because there’s a dedicated syntax for properties that is distinct from that for normal functions, the illusion of objects as actual things is strengthened for client code. ➤➤ Constructors are special functions that are called automatically when an object is instantiated. They must have the same name as the class to which they belong and cannot have a return type. Constructors are useful for initialization. ➤➤ Finalizers are similar to constructors but are called when the CLR detects that an object is no longer needed. They have the same name as the class, preceded by a tilde (~). It is impossible to predict precisely when a finalizer will be called. Finalizers are discussed in Chapter 14, “Memory Management and Pointers.” ➤➤ Operators, at their simplest, are actions such as + or –. When you add two integers, you are, strictly speaking, using the + operator for integers. However, C# also allows you to specify how existing operators will work with your own classes (operator overloading). Chapter 7, “Operators and Casts,” looks at operators in detail. ➤➤ Indexers allow your objects to be indexed in the same way as an array or collection. c03.indd 67 30-01-2014 20:05:39 68  ❘  CHAPTER 3  Objects and Types Methods Note that official C# terminology makes a distinction between functions and methods. In C# terminology, the term “function member” includes not only methods, but also other nondata members of a class or struct. This includes indexers, operators, constructors, destructors, and — perhaps somewhat surprisingly — properties. These are contrasted with data members: fields, constants, and events. Declaring Methods In C#, the definition of a method consists of any method modifiers (such as the method’s accessibility), followed by the type of the return value, followed by the name of the method, followed by a list of input arguments enclosed in parentheses, followed by the body of the method enclosed in curly braces: [modifiers] return_type MethodName([parameters]) { // Method body } Each parameter consists of the name of the type of the parameter, and the name by which it can be referenced in the body of the method. Also, if the method returns a value, a return statement must be used with the return value to indicate each exit point, as shown in this example: public bool IsSquare(Rectangle rect) { return (rect.Height == rect.Width); } This code uses one of the .NET base classes, System.Drawing.Rectangle, which represents a rectangle. If the method doesn’t return anything, specify a return type of void because you can’t omit the return type altogether; and if it takes no arguments, you still need to include an empty set of parentheses after the method name. In this case, including a return statement is optional — the method returns automatically when the closing curly brace is reached. Note that a method can contain as many return statements as required: public bool IsPositive(int value) { if (value < 0) return false; return true; } Invoking Methods The following example, MathTest, illustrates the syntax for definition and instantiation of classes, and definition and invocation of methods. Besides the class that contains the Main() method, it defines a class named MathTest, which contains a couple of methods and a field: using System; namespace Wrox { class MainEntryPoint { static void Main() { // Try calling some static functions. Console.WriteLine("Pi is " + MathTest.GetPi()); int x = MathTest.GetSquareOf(5); Console.WriteLine("Square of 5 is " + x); // Instantiate a MathTest object MathTest math = new MathTest(); // this is C#'s way of c03.indd 68 30-01-2014 20:05:39 Classes  ❘  69 // instantiating a reference type // Call nonstatic methods math.value = 30; Console.WriteLine( "Value field of math variable contains " + math.value); Console.WriteLine("Square of 30 is " + math.GetSquare()); } } // Define a class named MathTest on which we will call a method class MathTest { public int value; public int GetSquare() { return value*value; } public static int GetSquareOf(int x) { return x*x; } public static double GetPi() { return 3.14159; } } } Running the MathTest example produces the following results: Pi is 3.14159 Square of 5 is 25 Value field of math variable contains 30 Square of 30 is 900 As you can see from the code, the MathTest class contains a field that contains a number, as well as a method to find the square of this number. It also contains two static methods: one to return the value of pi and one to find the square of the number passed in as a parameter. Some features of this class are not really good examples of C# program design. For example, GetPi() would usually be implemented as a const field, but following good design here would mean using some concepts that have not yet been introduced. Passing Parameters to Methods In general, parameters can be passed into methods by reference or by value. When a variable is passed by reference, the called method gets the actual variable, or more to the point, a pointer to the variable in memory. Any changes made to the variable inside the method persist when the method exits. However, when a variable is passed by value, the called method gets an identical copy of the variable, meaning any changes made are lost when the method exits. For complex data types, passing by reference is more efficient because of the large amount of data that must be copied when passing by value. In C#, reference types are passed by reference and value types are passed by value unless you specify otherwise. However, be sure you understand the implications of this for reference types. Because reference type variables hold only a reference to an object, it is this reference that is passed in as a parameter, not the object itself. Hence, changes made to the underlying object will persist. Value type variables, in contrast, hold the actual data, so a copy of the data itself is passed into the method. An int, for instance, is passed by value to a method, and any changes that the method makes to the value of that int do not change the value c03.indd 69 30-01-2014 20:05:39 70  ❘  CHAPTER 3  Objects and Types of the original int object. Conversely, if an array or any other reference type, such as a class, is passed into a method, and the method uses the reference to change a value in that array, the new value is reflected in the original array object. Here is an example, ParameterTest.cs, which demonstrates the difference between value types and reference types used as parameters: using System; namespace Wrox { class ParameterTest { static void SomeFunction(int[] ints, int i) { ints[0] = 100; i = 100; } public static int Main() { int i = 0; int[] ints = { 0, 1, 2, 4, 8 }; // Display the original values. Console.WriteLine("i = " + i); Console.WriteLine("ints[0] = " + ints[0]); Console.WriteLine("Calling SomeFunction. .."); // After this method returns, ints will be changed, // but i will not. SomeFunction(ints, i); Console.WriteLine("i = " + i); Console.WriteLine("ints[0] = " + ints[0]); return 0; } } } The output of the preceding is as follows: ParameterTest.exe i = 0 ints[0] = 0 Calling SomeFunction ... i = 0 ints[0] = 100 Notice how the value of i remains unchanged, but the value changed in ints is also changed in the original array. The behavior of strings is different again. This is because strings are immutable (if you alter a string’s value, you create an entirely new string), so strings don’t display the typical reference-type behavior. Any changes made to a string within a method call won’t affect the original string. This point is discussed in more detail in Chapter 9, “Strings and Regular Expressions.” ref Parameters As mentioned, passing variables by value is the default, but you can force value parameters to be passed by reference. To do so, use the ref keyword. If a parameter is passed to a method, and the input argument for that method is prefixed with the ref keyword, any changes that the method makes to the variable will affect the value of the original object: c03.indd 70 30-01-2014 20:05:39 Classes  ❘  71 static void SomeFunction(int[] ints, ref int i) { ints[0] = 100; i = 100; // The change to i will persist after SomeFunction() exits. } You also need to add the ref keyword when you invoke the method: SomeFunction(ints, ref i); Finally, it is important to understand that C# continues to apply initialization requirements to parameters passed to methods. Any variable must be initialized before it is passed into a method, whether it is passed in by value or by reference. out Parameters In C-style languages, it is common for functions to be able to output more than one value from a single routine. This is accomplished using output parameters, by assigning the output values to variables that have been passed to the method by reference. Often, the starting values of the variables that are passed by reference are unimportant. Those values will be overwritten by the function, which may never even look at any previous value. It would be convenient if you could use the same convention in C#, but C# requires that variables be initialized with a starting value before they are referenced. Although you could initialize your input variables with meaningless values before passing them into a function that will fill them with real, meaningful ones, this practice is at best needless and at worst confusing. However, there is a way to circumvent the C# compiler’s insistence on initial values for input arguments. You do this with the out keyword. When a method’s input argument is prefixed with out, that method can be passed a variable that has not been initialized. The variable is passed by reference, so any changes that the method makes to the variable will persist when control returns from the called method. Again, you must use the out keyword when you call the method, as well as when you define it: static void SomeFunction(out int i) { i = 100; } public static int Main() { int i; // note how i is declared but not initialized. SomeFunction(out i); Console.WriteLine(i); return 0; } Named Arguments Typically, parameters need to be passed into a method in the same order that they are defined. Named arguments allow you to pass in parameters in any order. So for the following method: string FullName(string firstName, string lastName) { return firstName + " " + lastName; } The following method calls will return the same full name: FullName("John", "Doe"); FullName(lastName: "Doe", firstName: "John"); If the method has several parameters, you can mix positional and named arguments in the same call. c03.indd 71 30-01-2014 20:05:39 72  ❘  CHAPTER 3  Objects and Types Optional Arguments Parameters can also be optional. You must supply a default value for optional parameters, which must be the last ones defined. For example, the following method declaration would be incorrect: void TestMethod(int optionalNumber = 10, int notOptionalNumber) { System.Console.Write(optionalNumber + notOptionalNumber); } For this method to work, the optionalNumber parameter would have to be defined last. Method Overloading C# supports method overloading — several versions of the method that have different signatures (that is, the same name but a different number of parameters and/or different parameter data types). To overload methods, simply declare the methods with the same name but different numbers or types of parameters: class ResultDisplayer { void DisplayResult(string result) { // implementation } void DisplayResult(int result) { // implementation } } If optional parameters won’t work for you, then you need to use method overloading to achieve the same effect: class MyClass { int DoSomething(int x) // want 2nd parameter with default value 10 { DoSomething(x, 10); } int DoSomething(int x, int y) { // implementation } } As in any language, method overloading carries with it the potential for subtle runtime bugs if the wrong overload is called. Chapter 4 discusses how to code defensively against these problems. For now, you should know that C# does place some minimum restrictions on the parameters of overloaded methods: ➤➤ It is not sufficient for two methods to differ only in their return type. ➤➤ It is not sufficient for two methods to differ only by virtue of a parameter having been declared as ref or out. Properties The idea of a property is that it is a method or a pair of methods dressed to look like a field. A good example of this is the Height property of a Windows form. Suppose that you have the following code: // mainForm is of type System.Windows.Forms mainForm.Height = 400; c03.indd 72 30-01-2014 20:05:39 Classes  ❘  73 On executing this code, the height of the window will be set to 400 px, and you will see the window resize on the screen. Syntactically, this code looks like you’re setting a field, but in fact you are calling a property accessor that contains code to resize the form. To define a property in C#, use the following syntax: public string SomeProperty { get { return "This is the property value."; } set { // do whatever needs to be done to set the property. } } The get accessor takes no parameters and must return the same type as the declared property. You should not specify any explicit parameters for the set accessor either, but the compiler assumes it takes one parameter, which is of the same type again, and which is referred to as value. For example, the following code contains a property called Age, which sets a field called age. In this example, age is referred to as the backing variable for the property Age: private int age; public int Age { get { return age; } set { age = value; } } Note the naming convention used here. You take advantage of C#’s case sensitivity by using the same name, Pascal-case for the public property, and camel-case for the equivalent private field if there is one. Some developers prefer to use field names that are prefixed by an underscore: _age; this provides an extremely convenient way to identify fields. Read-Only and Write-Only Properties It is possible to create a read-only property by simply omitting the set accessor from the property definition. Thus, to make Name a read-only property, you would do the following: private string name; public string Name { get { return name; } } It is similarly possible to create a write-only property by omitting the get accessor. However, this is regarded as poor programming practice because it could be confusing to authors of client code. In general, it is recommended that if you are tempted to do this, you should use a method instead. c03.indd 73 30-01-2014 20:05:39 74 ❘ CHAPTER 3 Objects and types Access Modifi ers for Properties C# does allow the set and get accessors to have differing access modifi ers. This would allow a property to have a public get and a private or protected set. This can help control how or when a property can be set. In the following code example, notice that the set has a private access modifi er but the get does not. In this case, the get takes the access level of the property. One of the accessors must follow the access level of the property. A compile error will be generated if the get accessor has the protected access level associated with it because that would make both accessors have a different access level from the property. public string Name { get { return _name; } private set { _name = value; } } Auto-Implemented Properties If there isn’t going to be any logic in the properties set and get, then auto-implemented properties can be used. Auto-implemented properties implement the backing member variable automatically. The code for the earlier Age example would look like this: public int Age {get; set;} The declaration private int Age; is not needed. The compiler will create this automatically. By using auto-implemented properties, validation of the property cannot be done at the property set. Therefore, in the last example you could not have checked to see if an invalid age is set. Also, both accessors must be present, so an attempt to make a property read-only would cause an error: public int Age {get;} However, the access level of each accessor can be different, so the following is acceptable: public int Age {get; private set;} A NoTE AbouT INlINING Some developers may be concerned that the previous sections have presented a number of situations in which standard C# coding practices have led to very small functions — for example, accessing a fi eld via a property instead of directly. Will this hurt performance because of the overhead of the extra function call? The answer is no. There’s no need to worry about performance loss from these kinds of programming methodologies in C#. Recall that C# code is compiled to IL, then JIT compiled at runtime to native executable code. The JIT compiler is designed to generate highly optimized code and will ruthlessly inline code as appropriate (in other words, it replaces function calls with inline code). A method or property whose implementation simply calls another method or returns a fi eld will almost certainly be inlined. However, the decision regarding where to inline is made entirely by the CLR. You cannot control which methods are inlined by using, for example, a keyword similar to the inline keyword of C++. c03.indd 74 30-01-2014 20:05:40 Classes  ❘  75 Constructors The syntax for declaring basic constructors is a method that has the same name as the containing class and that does not have any return type: public class MyClass { public MyClass() { } // rest of class definition It’s not necessary to provide a constructor for your class. We haven’t supplied one for any of the examples so far in this book. In general, if you don’t supply any constructor, the compiler will generate a default one behind the scenes. It will be a very basic constructor that just initializes all the member fields by zeroing them out (null reference for reference types, zero for numeric data types, and false for bools). Often, that will be adequate; if not, you’ll need to write your own constructor. Constructors follow the same rules for overloading as other methods — that is, you can provide as many overloads to the constructor as you want, provided they are clearly different in signature: public MyClass() // zeroparameter constructor { // construction code } public MyClass(int number) // another overload { // construction code } However, if you supply any constructors that take parameters, the compiler will not automatically supply a default one. This is done only if you have not defined any constructors at all. In the following example, because a one-parameter constructor is defined, the compiler assumes that this is the only constructor you want to be available, so it will not implicitly supply any others: public class MyNumber { private int number; public MyNumber(int number) { this.number = number; } } This code also illustrates typical use of the this keyword to distinguish member fields from parameters of the same name. If you now try instantiating a MyNumber object using a no-parameter constructor, you will get a compilation error: MyNumber numb = new MyNumber(); // causes compilation error Note that it is possible to define constructors as private or protected, so that they are invisible to code in unrelated classes too: public class MyNumber { private int number; private MyNumber(int number) // another overload { this.number = number; } } This example hasn’t actually defined any public or even any protected constructors for MyNumber. This would actually make it impossible for MyNumber to be instantiated by outside code using the new operator c03.indd 75 30-01-2014 20:05:40 76  ❘  CHAPTER 3  Objects and Types (though you might write a public static property or method in MyNumber that can instantiate the class). This is useful in two situations: ➤➤ If your class serves only as a container for some static members or properties, and therefore should never be instantiated ➤➤ If you want the class to only ever be instantiated by calling a static member function (this is the so-called “class factory” approach to object instantiation) Static Constructors One novel feature of C# is that it is also possible to write a static no-parameter constructor for a class. Such a constructor is executed only once, unlike the constructors written so far, which are instance constructors that are executed whenever an object of that class is created: class MyClass { static MyClass() { // initialization code } // rest of class definition } One reason for writing a static constructor is if your class has some static fields or properties that need to be initialized from an external source before the class is first used. The .NET runtime makes no guarantees about when a static constructor will be executed, so you should not place any code in it that relies on it being executed at a particular time (for example, when an assembly is loaded). Nor is it possible to predict in what order static constructors of different classes will execute. However, what is guaranteed is that the static constructor will run at most once, and that it will be invoked before your code makes any reference to the class. In C#, the static constructor is usually executed immediately before the first call, to any member of the class. Note that the static constructor does not have any access modifiers. It’s never called by any other C# code, but always by the .NET runtime when the class is loaded, so any access modifier such as public or private would be meaningless. For this same reason, the static constructor can never take any parameters, and there can be only one static constructor for a class. It should also be obvious that a static constructor can access only static members, not instance members, of the class. It is possible to have a static constructor and a zero-parameter instance constructor defined in the same class. Although the parameter lists are identical, there is no conflict because the static constructor is executed when the class is loaded, but the instance constructor is executed whenever an instance is created. Therefore, there is no confusion about which constructor is executed or when. If you have more than one class that has a static constructor, the static constructor that will be executed first is undefined. Therefore, you should not put any code in a static constructor that depends on other static constructors having been or not having been executed. However, if any static fields have been given default values, these will be allocated before the static constructor is called. The next example illustrates the use of a static constructor. It is based on the idea of a program that has user preferences (which are presumably stored in some configuration file). To keep things simple, assume just one user preference, a quantity called BackColor that might represent the background color to be used in an application. Because we don’t want to get into the details of writing code to read data from an external source here, assume also that the preference is to have a background color of red on weekdays and green on weekends. All the program does is display the preference in a console window, but that is enough to see a static constructor at work: namespace Wrox.ProCSharp.StaticConstructorSample { public class UserPreferences { c03.indd 76 30-01-2014 20:05:40 Classes  ❘  77 public static readonly Color BackColor; static UserPreferences() { DateTime now = DateTime.Now; if (now.DayOfWeek == DayOfWeek.Saturday || now.DayOfWeek == DayOfWeek.Sunday) BackColor = Color.Green; else BackColor = Color.Red; } private UserPreferences() { } } } This code shows how the color preference is stored in a static variable, which is initialized in the static constructor. The field is declared as read-only, which means that its value can only be set in a constructor. You learn about read-only fields in more detail later in this chapter. The code uses a few helpful structs that Microsoft has supplied as part of the Framework class library. System.DateTime and System.Drawing .Color. DateTime implement a static property, Now, which returns the current time; and an instance property, DayOfWeek, which determines what day of the week a date-time represents. Color is used to store colors. It implements various static properties, such as Red and Green as used in this example, which return commonly used colors. To use Color, you need to reference the System.Drawing.dll assembly when compiling, and you must add a using statement for the System.Drawing namespace: using System; using System.Drawing; You test the static constructor with this code: class MainEntryPoint { static void Main(string[] args) { Console.WriteLine("User-preferences: BackColor is: " + UserPreferences.BackColor.ToString()); } } Compiling and running the preceding code results in this output: User-preferences: BackColor is: Color [Red] Of course, if the code is executed during the weekend, your color preference would be Green. Calling Constructors from Other Constructors You might sometimes find yourself in the situation where you have several constructors in a class, perhaps to accommodate some optional parameters for which the constructors have some code in common. For example, consider the following: class Car { private string description; private uint nWheels; public Car(string description, uint nWheels) { this.description = description; c03.indd 77 30-01-2014 20:05:40 78  ❘  CHAPTER 3  Objects and Types this.nWheels = nWheels; } public Car(string description) { this.description = description; this.nWheels = 4; } // etc. Both constructors initialize the same fields. It would clearly be neater to place all the code in one location. C# has a special syntax known as a constructor initializer to enable this: class Car { private string description; private uint nWheels; public Car(string description, uint nWheels) { this.description = description; this.nWheels = nWheels; } public Car(string description): this(description, 4) { } // etc In this context, the this keyword simply causes the constructor with the nearest matching parameters to be called. Note that any constructor initializer is executed before the body of the constructor. Suppose that the following code is run: Car myCar = new Car("Proton Persona"); In this example, the two-parameter constructor executes before any code in the body of the one-parameter constructor (though in this particular case, because there is no code in the body of the one-parameter constructor, it makes no difference). A C# constructor initializer may contain either one call to another constructor in the same class (using the syntax just presented) or one call to a constructor in the immediate base class (using the same syntax, but using the keyword base instead of this). It is not possible to put more than one call in the initializer. readonly Fields The concept of a constant as a variable that contains a value that cannot be changed is something that C# shares with most programming languages. However, constants don’t necessarily meet all requirements. On occasion, you may have a variable whose value shouldn’t be changed but the value is not known until runtime. C# provides another type of variable that is useful in this scenario: the readonly field. The readonly keyword provides a bit more flexibility than const, allowing for situations in which you want a field to be constant but you also need to carry out some calculations to determine its initial value. The rule is that you can assign values to a readonly field inside a constructor, but not anywhere else. It’s also possible for a readonly field to be an instance rather than a static field, having a different value for each instance of a class. This means that, unlike a const field, if you want a readonly field to be static, you have to declare it as such. Suppose that you have an MDI program that edits documents, and for licensing reasons you want to restrict the number of documents that can be opened simultaneously. Assume also that you are selling different versions of the software, and it’s possible for customers to upgrade their licenses to open more documents simultaneously. Clearly, this means you can’t hard-code the maximum number in the source code. You would probably need a field to represent this maximum number. This field will have to be read in — perhaps c03.indd 78 30-01-2014 20:05:40 Anonymous Types  ❘  79 from a registry key or some other file storage — each time the program is launched. Therefore, your code might look something like this: public class DocumentEditor { public static readonly uint MaxDocuments; static DocumentEditor() { MaxDocuments = DoSomethingToFindOutMaxNumber(); } } In this case, the field is static because the maximum number of documents needs to be stored only once per running instance of the program. This is why it is initialized in the static constructor. If you had an instance readonly field, you would initialize it in the instance constructor(s). For example, presumably each document you edit has a creation date, which you wouldn’t want to allow the user to change (because that would be rewriting the past!). Note that the field is also public — you don’t normally need to make readonly fields private, because by definition they cannot be modified externally (the same principle also applies to constants). As noted earlier, date is represented by the class System.DateTime. The following code uses a System .DateTime constructor that takes three parameters (year, month, and day of the month; for details about this and other DateTime constructors see the MSDN documentation): public class Document { public readonly DateTime CreationDate; public Document() { // Read in creation date from file. Assume result is 1 Jan 2002 // but in general this can be different for different instances // of the class CreationDate = new DateTime(2002, 1, 1); } } CreationDate and MaxDocuments in the previous code snippet are treated like any other field, except that because they are read-only they cannot be assigned outside the constructors: void SomeMethod() { MaxDocuments = 10; // compilation error here. MaxDocuments is readonly } It’s also worth noting that you don’t have to assign a value to a readonly field in a constructor. If you don’t do so, it will be left with the default value for its particular data type or whatever value you initialized it to at its declaration. That applies to both static and instance readonly fields. Anonymous Types Chapter 2, “Core C#” discussed the var keyword in reference to implicitly typed variables. When used with the new keyword, anonymous types can be created. An anonymous type is simply a nameless class that inherits from object. The definition of the class is inferred from the initializer, just as with implicitly typed variables. For example, if you needed an object containing a person’s first, middle, and last name, the declaration would look like this: var captain = new {FirstName = "James", MiddleName = "T", LastName = "Kirk"}; c03.indd 79 30-01-2014 20:05:41 80  ❘  CHAPTER 3  Objects and Types This would produce an object with FirstName, MiddleName, and LastName properties. If you were to create another object that looked like: var doctor = new {FirstName = "Leonard", MiddleName = "", LastName = "McCoy"}; then the types of captain and doctor are the same. You could set captain = doctor, for example. If the values that are being set come from another object, then the initializer can be abbreviated. If you already have a class that contains the properties FirstName, MiddleName, and LastName and you have an instance of that class with the instance name person, then the captain object could be initialized like this: var captain = new {person.FirstName, person.MiddleName, person.LastName}; The property names from the person object would be projected to the new object named captain, so the object named captain would have the FirstName, MiddleName, and LastName properties. The actual type name of these new objects is unknown. The compiler “makes up” a name for the type, but only the compiler is ever able to make use of it. Therefore, you can’t and shouldn’t plan on using any type reflection on the new objects because you will not get consistent results. Structs So far, you have seen how classes offer a great way to encapsulate objects in your program. You have also seen how they are stored on the heap in a way that gives you much more flexibility in data lifetime, but with a slight cost in performance. This performance cost is small thanks to the optimizations of managed heaps. However, in some situations all you really need is a small data structure. If so, a class provides more functionality than you need, and for best performance you probably want to use a struct. Consider the following example: class Dimensions { public double Length; public double Width; } This code defines a class called Dimensions, which simply stores the length and width of an item. Suppose you’re writing a furniture-arranging program that enables users to experiment with rearranging their furniture on the computer, and you want to store the dimensions of each item of furniture. It might seem as though you’re breaking the rules of good program design by making the fields public, but the point is that you don’t really need all the facilities of a class for this. All you have is two numbers, which you’ll find convenient to treat as a pair rather than individually. There is no need for a lot of methods, or for you to be able to inherit from the class, and you certainly don’t want to have the .NET runtime go to the trouble of bringing in the heap, with all the performance implications, just to store two doubles. As mentioned earlier in this chapter, the only thing you need to change in the code to define a type as a struct instead of a class is to replace the keyword class with struct: struct Dimensions { public double Length; public double Width; } Defining functions for structs is also exactly the same as defining them for classes. The following code demonstrates a constructor and a property for a struct: struct Dimensions { public double Length; public double Width; public Dimensions(double length, double width) c03.indd 80 30-01-2014 20:05:41 Structs  ❘  81 { Length = length; Width = width; } public double Diagonal { get { return Math.Sqrt(Length * Length + Width * Width); } } } Structs are value types, not reference types. This means they are stored either in the stack or inline (if they are part of another object that is stored on the heap) and have the same lifetime restrictions as the simple data types: ➤➤ Structs do not support inheritance. ➤➤ There are some differences in the way constructors work for structs. In particular, the compiler always supplies a default no-parameter constructor, which you are not permitted to replace. ➤➤ With a struct, you can specify how the fields are to be laid out in memory (this is examined in Chapter 15, “Reflection,” which covers attributes). Because structs are really intended to group data items together, you’ll sometimes find that most or all of their fields are declared as public. Strictly speaking, this is contrary to the guidelines for writing .NET code — according to Microsoft, fields (other than const fields) should always be private and wrapped by public properties. However, for simple structs, many developers consider public fields to be acceptable programming practice. The following sections look at some of these differences between structs and classes in more detail. Structs Are Value Types Although structs are value types, you can often treat them syntactically in the same way as classes. For example, with the definition of the Dimensions class in the previous section, you could write this: Dimensions point = new Dimensions(); point.Length = 3; point.Width = 6; Note that because structs are value types, the new operator does not work in the same way as it does for classes and other reference types. Instead of allocating memory on the heap, the new operator simply calls the appropriate constructor, according to the parameters passed to it, initializing all fields. Indeed, for structs it is perfectly legal to write this: Dimensions point; point.Length = 3; point.Width = 6; If Dimensions were a class, this would produce a compilation error, because point would contain an uninitialized reference — an address that points nowhere, so you could not start setting values to its fields. For a struct, however, the variable declaration actually allocates space on the stack for the entire struct, so it’s ready to assign values to. The following code, however, would cause a compilation error, with the compiler complaining that you are using an uninitialized variable: Dimensions point; Double D = point.Length; Structs follow the same rule as any other data type — everything must be initialized before use. A struct is considered fully initialized either when the new operator has been called against it or when values have c03.indd 81 30-01-2014 20:05:41 82  ❘  CHAPTER 3  Objects and Types been individually assigned to all its fields. Also, of course, a struct defined as a member field of a class is initialized by being zeroed out automatically when the containing object is initialized. The fact that structs are value types affects performance, though depending on how you use your struct, this can be good or bad. On the positive side, allocating memory for structs is very fast because this takes place inline or on the stack. The same is true when they go out of scope. Structs are cleaned up quickly and don’t need to wait on garbage collection. On the negative side, whenever you pass a struct as a parameter or assign a struct to another struct (as in A=B, where A and B are structs), the full contents of the struct are copied, whereas for a class only the reference is copied. This results in a performance loss that varies according to the size of the struct, emphasizing the fact that structs are really intended for small data structures. Note, however, that when passing a struct as a parameter to a method, you can avoid this performance loss by passing it as a ref parameter — in this case, only the address in memory of the struct will be passed in, which is just as fast as passing in a class. If you do this, though, be aware that it means the called method can, in principle, change the value of the struct. Structs and Inheritance Structs are not designed for inheritance. This means it is not possible to inherit from a struct. The only exception to this is that structs, in common with every other type in C#, derive ultimately from the class System.Object. Hence, structs also have access to the methods of System.Object, and it is even possible to override them in structs — an obvious example would be overriding the ToString() method. The actual inheritance chain for structs is that each struct derives from a class, System.ValueType, which in turn derives from System.Object. ValueType which does not add any new members to Object but provides implementations of some of them that are more suitable for structs. Note that you cannot supply a different base class for a struct: Every struct is derived from ValueType. Constructors for Structs You can define constructors for structs in exactly the same way that you can for classes, but you are not permitted to define a constructor that takes no parameters. This may seem nonsensical, but the reason is buried in the implementation of the .NET runtime. In some rare circumstances, the .NET runtime would not be able to call a custom zero-parameter constructor that you have supplied. Microsoft has therefore taken the easy way out and banned zero-parameter constructors for structs in C#. That said, the default constructor, which initializes all fields to zero values, is always present implicitly, even if you supply other constructors that take parameters. It’s also impossible to circumvent the default constructor by supplying initial values for fields. The following code will cause a compile-time error: struct Dimensions { public double Length = 1; // error. Initial values not allowed public double Width = 2; // error. Initial values not allowed } Of course, if Dimensions had been declared as a class, this code would have compiled without any problems. Incidentally, you can supply a Close() or Dispose() method for a struct in the same way you do for a class. The Dispose() method is discussed in detail in Chapter 14. Weak References When the class or struct is instantiated in the application code, it will have a strong reference as long as there is any other code that references it. For example, if you have a class called MyClass() and you create a reference to objects based on that class and call the variable myClassVariable as follows, as long as myClassVariable is in scope there is a strong reference to the MyClass object: MyClass myClassVariable = new MyClass(); c03.indd 82 30-01-2014 20:05:41 Partial Classes  ❘  83 This means that the garbage collector cannot clean up the memory used by the MyClass object. Generally this is a good thing because you may need to access the MyClass object; but what if MyClass were very large and perhaps wasn’t accessed very often? Then a weak reference to the object can be created. A weak reference allows the object to be created and used, but if the garbage collector happens to run (garbage collection is discussed in Chapter 14), it will collect the object and free up the memory. This is not something you would typically want to do because of potential bugs and performance issues, but there are certainly situations in which it makes sense. Weak references are created using the WeakReference class. Because the object could be collected at any time, it’s important that the existence of the object is valid before trying to reference it. Using the MathTest class from before, this time we’ll create a weak reference to it using the WeakReference class: static void Main() { // Instantiate a weak reference to MathTest object WeakReference mathReference = new WeakReference(new MathTest()); MathTest math; if(mathReference.IsAlive) { math = mathReference.Target as MathTest; math.Value = 30; Console.WriteLine("Value field of math variable contains " + math.Value); Console.WriteLine("Square of 30 is " + math.GetSquare()); } else { Console.WriteLine("Reference is not available."); } GC.Collect(); if(mathReference.IsAlive) { math = mathReference.Target as MathTest; } else { Console.WriteLine("Reference is not available."); } } When you create mathReference a new MathTest object is passed into the constructor. The MathTest object becomes the target of the WeakReference object. When you want to use the MathTest object, you have to check the mathReference object first to ensure it hasn’t been collected. You use the IsAlive property for that. If the IsAlive property is true, then you can get the reference to the MathTest object from the target property. Notice that you have to cast to the MathTest type, as the Target property returns an Object type. Next, you call the garbage collector (GC.Collect()) and try to get the MathTest object again. This time the IsAlive property returns false, and if you really wanted a MathTest object you would have to instantiate a new version. Partial Classes The partial keyword allows the class, struct, method, or interface to span multiple files. Typically, a class resides entirely in a single file. However, in situations in which multiple developers need access to the same class, or, more likely, a code generator of some type is generating part of a class, having the class in multiple files can be beneficial. c03.indd 83 30-01-2014 20:05:41 84  ❘  CHAPTER 3  Objects and Types To use the partial keyword, simply place partial before class, struct, or interface. In the following example, the class TheBigClass resides in two separate source files, BigClassPart1.cs and BigClassPart2.cs: //BigClassPart1.cs partial class TheBigClass { public void MethodOne() { } } //BigClassPart2.cs partial class TheBigClass { public void MethodTwo() { } } When the project that these two source files are part of is compiled, a single type called TheBigClass will be created with two methods, MethodOne() and MethodTwo(). If any of the following keywords are used in describing the class, the same must apply to all partials of the same type: ➤➤ public ➤➤ private ➤➤ protected ➤➤ internal ➤➤ abstract ➤➤ sealed ➤➤ new ➤➤ generic constraints Nested partials are allowed as long as the partial keyword precedes the class keyword in the nested type. Attributes, XML comments, interfaces, generic-type parameter attributes, and members are combined when the partial types are compiled into the type. Given these two source files: //BigClassPart1.cs [CustomAttribute] partial class TheBigClass: TheBigBaseClass, IBigClass { public void MethodOne() { } } //BigClassPart2.cs [AnotherAttribute] partial class TheBigClass: IOtherBigClass { public void MethodTwo() { } } the equivalent source file would be as follows after the compile: [CustomAttribute] [AnotherAttribute] c03.indd 84 30-01-2014 20:05:41 The Object Class  ❘  85 partial class TheBigClass: TheBigBaseClass, IBigClass, IOtherBigClass { public void MethodOne() { } public void MethodTwo() { } } Static Classes Earlier, this chapter discussed static constructors and how they allowed the initialization of static member variables. If a class contains nothing but static methods and properties, the class itself can become static. A static class is functionally the same as creating a class with a private static constructor. An instance of the class can never be created. By using the static keyword, the compiler can verify that instance members are never accidentally added to the class. If they are, a compile error occurs. This helps guarantee that an instance is never created. The syntax for a static class looks like this: static class StaticUtilities { public static void HelperMethod() { } } An object of type StaticUtilities is not needed to call the HelperMethod(). The type name is used to make the call: StaticUtilities.HelperMethod(); The Object Class As indicated earlier, all .NET classes are ultimately derived from System.Object. In fact, if you don’t specify a base class when you define a class, the compiler automatically assumes that it derives from Object. Because inheritance has not been used in this chapter, every class you have seen here is actually derived from System.Object. (As noted earlier, for structs this derivation is indirect — a struct is always derived from System.ValueType, which in turn derives from System.Object.) The practical significance of this is that, besides the methods, properties, and so on that you define, you also have access to a number of public and protected member methods that have been defined for the Object class. These methods are available in all other classes that you define. System.Object Methods For the time being, the following list summarizes the purpose of each method; the next section provides more details about the ToString() method in particular: ➤➤ ToString() —  A fairly basic, quick-and-easy string representation. Use it when you just want a quick idea of the contents of an object, perhaps for debugging purposes. It provides very little choice regarding how to format the data. For example, dates can, in principle, be expressed in a huge variety of different formats, but DateTime.ToString() does not offer you any choice in this regard. If you need a more sophisticated string representation — for example, one that takes into account your formatting preferences or the culture (the locale) — then you should implement the IFormattable interface (see Chapter 9). ➤➤ GetHashCode() —  If objects are placed in a data structure known as a map (also known as a hash table or dictionary), it is used by classes that manipulate these structures to determine where to place c03.indd 85 30-01-2014 20:05:42 86  ❘  CHAPTER 3  Objects and Types an object in the structure. If you intend your class to be used as a key for a dictionary, you need to override GetHashCode(). Some fairly strict requirements exist for how you implement your overload, which you learn about when you examine dictionaries in Chapter 10, “Collections.” ➤➤ Equals() (both versions) and ReferenceEquals() —  As you’ll note by the existence of three different methods aimed at comparing the equality of objects, the .NET Framework has quite a sophisticated scheme for measuring equality. Subtle differences exist between how these three methods, along with the comparison operator, ==, are intended to be used. In addition, restrictions exist on how you should override the virtual, one-parameter version of Equals() if you choose to do so, because certain base classes in the System.Collections namespace call the method and expect it to behave in certain ways. You explore the use of these methods in Chapter 7 when you examine operators. ➤➤ Finalize() —  Covered in Chapter 13, “Asynchronous Programming,” this method is intended as the nearest that C# has to C++-style destructors. It is called when a reference object is garbage collected to clean up resources. The Object implementation of Finalize()doesn’t actually do anything and is ignored by the garbage collector. You normally override Finalize() if an object owns references to unmanaged resources that need to be removed when the object is deleted. The garbage collector cannot do this directly because it only knows about managed resources, so it relies on any finalizers that you supply. ➤➤ GetType() —  This object returns an instance of a class derived from System.Type, so it can provide an extensive range of information about the class of which your object is a member, including base type, methods, properties, and so on. System.Type also provides the entry point into .NET’s reflection technology. Chapter 15 examines this topic. ➤➤ MemberwiseClone() —  The only member of System.Object that isn’t examined in detail anywhere in the book. That’s because it is fairly simple in concept. It just makes a copy of the object and returns a reference (or in the case of a value type, a boxed reference) to the copy. Note that the copy made is a shallow copy, meaning it copies all the value types in the class. If the class contains any embedded references, then only the references are copied, not the objects referred to. This method is protected and cannot be called to copy external objects. Nor is it virtual, so you cannot override its implementation. The ToString() Method You’ve already encountered ToString() in Chapter 2. It provides the most convenient way to get a quick string representation of an object. For example: int i = 50; string str = i.ToString(); // returns "50" Here’s another example: enum Colors {Red, Orange, Yellow}; // later on in code... Colors favoriteColor = Colors.Orange; string str = favoriteColor.ToString(); // returns "Orange" Object.ToString() is actually declared as virtual, and all these examples are taking advantage of the fact that its implementation in the C# predefined data types has been overridden for us to return correct string representations of those types. You might not think that the Colors enum counts as a predefined data type. It actually is implemented as a struct derived from System.Enum, and System.Enum has a rather clever override of ToString() that deals with all the enums you define. If you don’t override ToString() in classes that you define, your classes will simply inherit the System .Object implementation, which displays the name of the class. If you want ToString() to return a string that contains information about the value of objects of your class, you need to override it. To illustrate this, the following example, Money, defines a very simple class, also called Money, which represents U.S. currency c03.indd 86 30-01-2014 20:05:42 Extension Methods  ❘  87 amounts. Money simply acts as a wrapper for the decimal class but supplies a ToString() method. Note that this method must be declared as override because it is replacing (overriding) the ToString() method supplied by Object. Chapter 4 discusses overriding in more detail. The complete code for this example is as follows (note that it also illustrates use of properties to wrap fields): using System; namespace Wrox { class MainEntryPoint { static void Main(string[] args) { Money cash1 = new Money(); cash1.Amount = 40M; Console.WriteLine("cash1.ToString() returns: " + cash1.ToString()); Console.ReadLine(); } } public class Money { private decimal amount; public decimal Amount { get { return amount; } set { amount = value; } } public override string ToString() { return "$" + Amount.ToString(); } } } This example is included just to illustrate syntactical features of C#. C# already has a predefined type to represent currency amounts, decimal, so in real life you wouldn’t write a class to duplicate this functionality unless you wanted to add various other methods to it; and in many cases, due to formatting requirements, you’d probably use the String.Format() method (which is covered in Chapter 8) rather than ToString() to display a currency string. In the Main() method, you first instantiate a Money object. The ToString() method is then called, which actually executes the overridden version of the method. Running this code gives the following results: cash1.ToString() returns: $40 Extension Methods There are many ways to extend a class. If you have the source for the class, then inheritance, which is covered in Chapter 4, is a great way to add functionality to your objects. If the source code isn’t available, extension methods can help by enabling you to change a class without requiring the source code for the class. Extension methods are static methods that can appear to be part of a class without actually being in the source code for the class. Let’s say that the Money class from the previous example needs to have a method AddToAmount(decimal amountToAdd). However, for whatever reason, the original source for the assembly c03.indd 87 30-01-2014 20:05:42 88  ❘  CHAPTER 3  Objects and Types cannot be changed directly. All you have to do is create a static class and add the AddToAmount method as a static method. Here is what the code would look like: namespace Wrox { public static class MoneyExtension { public static void AddToAmount(this Money money, decimal amountToAdd) { money.Amount += amountToAdd; } } } Notice the parameters for the AddToAmount method. For an extension method, the first parameter is the type that is being extended preceded by the this keyword. This is what tells the compiler that this method is part of the Money type. In this example, Money is the type that is being extended. In the extension method you have access to all the public methods and properties of the type being extended. In the main program, the AddToAmount method appears just as another method. The first parameter doesn’t appear, and you do not have to do anything with it. To use the new method, you make the call just like any other method: cash1.AddToAmount(10M); Even though the extension method is static, you use standard instance method syntax. Notice that you call AddToAmount using the cash1 instance variable and not using the type name. If the extension method has the same name as a method in the class, the extension method will never be called. Any instance methods already in the class take precedence. Summary This chapter examined C# syntax for declaring and manipulating objects. You have seen how to declare static and instance fields, properties, methods, and constructors. You have also seen that C# adds some new features not present in the OOP model of some other languages — for example, static constructors provide a means of initializing static fields, whereas structs enable you to define types that do not require the use of the managed heap, which could result in performance gains. You have also seen how all types in C# derive ultimately from the type System.Object, which means that all types start with a basic set of useful methods, including ToString(). We mentioned inheritance a few times throughout this chapter, and you’ll examine implementation and interface inheritance in C# in Chapter 4. c03.indd 88 30-01-2014 20:05:42 Inheritance WHAT’S In THIS CHAPTER? ➤➤ Types of inheritance ➤➤ Implementing inheritance ➤➤ Access modifi ers ➤➤ Interfaces WRoX.CoM CoDE DoWnloADS FoR THIS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ BankAccounts.cs ➤➤ CurrentAccounts.cs ➤➤ MortimerPhones.cs InHERITAnCE Chapter 3, “Objects and Types,” examined how to use individual classes in C#. The focus in that chapter was how to defi ne methods, properties, constructors, and other members of a single class (or a single struct). Although you learned that all classes are ultimately derived from the class System .Object, you have not yet learned how to create a hierarchy of inherited classes. Inheritance is the subject of this chapter, which explains how C# and the .NET Framework handle inheritance. TyPES oF InHERITAnCE Let’s start by reviewing exactly what C# does and does not support as far as inheritance is concerned. 4 c04.indd 89 30-01-2014 20:06:28 90  ❘  CHAPTER 4  Inheritance Implementation Versus Interface Inheritance In object-oriented programming, there are two distinct types of inheritance — implementation inheritance and interface inheritance: ➤➤ Implementation inheritance means that a type derives from a base type, taking all the base type’s member fields and functions. With implementation inheritance, a derived type adopts the base type’s implementation of each function, unless the definition of the derived type indicates that a function implementation is to be overridden. This type of inheritance is most useful when you need to add functionality to an existing type, or when a number of related types share a significant amount of common functionality. ➤➤ Interface inheritance means that a type inherits only the signatures of the functions, not any implementations. This type of inheritance is most useful when you want to specify that a type makes certain features available. C# supports both implementation inheritance and interface inheritance. Both are incorporated into the framework and the language from the ground up, thereby enabling you to decide which to use based on the application’s architecture. Multiple Inheritance Some languages such as C++ support what is known as multiple inheritance, in which a class derives from more than one other class. The benefits of using multiple inheritance are debatable: On the one hand, you can certainly use multiple inheritance to write extremely sophisticated, yet compact code, as demonstrated by the C++ ATL library. On the other hand, code that uses multiple implementation inheritance is often difficult to understand and debug (a point that is equally well demonstrated by the C++ ATL library). As mentioned, making it easy to write robust code was one of the crucial design goals behind the development of C#. Accordingly, C# does not support multiple implementation inheritance. It does, however, allow types to be derived from multiple interfaces — multiple interface inheritance. This means that a C# class can be derived from one other class, and any number of interfaces. Indeed, we can be more precise: Thanks to the presence of System.Object as a common base type, every C# class (except for Object) has exactly one base class, and may additionally have any number of base interfaces. Structs and Classes Chapter 3 distinguishes between structs (value types) and classes (reference types). One restriction of using structs is that they do not support inheritance, beyond the fact that every struct is automatically derived from System.ValueType. Although it’s true that you cannot code a type hierarchy of structs, it is possible for structs to implement interfaces. In other words, structs don’t really support implementation inheritance, but they do support interface inheritance. The following summarizes the situation for any types that you define: ➤➤ Structs are always derived from System.ValueType. They can also be derived from any number of interfaces. ➤➤ Classes are always derived from either System.Object or one that you choose. They can also be derived from any number of interfaces. Implementation Inheritance If you want to declare that a class derives from another class, use the following syntax: class MyDerivedClass: MyBaseClass { // functions and data members here } c04.indd 90 30-01-2014 20:06:28 Implementation Inheritance  ❘  91 Note  This syntax is very similar to C++ and Java syntax. However, C++ programmers, who will be used to the concepts of public and private inheritance, should note that C# does not support private inheritance, hence the absence of a public or private qualifier on the base class name. Supporting private inheritance would have complicated the language for very little gain. In practice, private inheritance is very rarely in C++ anyway. If a class (or a struct) also derives from interfaces, the list of base class and interfaces is separated by commas: public class MyDerivedClass: MyBaseClass, IInterface1, IInterface2 { // etc. } For a struct, the syntax is as follows: public struct MyDerivedStruct: IInterface1, IInterface2 { // etc. } If you do not specify a base class in a class definition, the C# compiler will assume that System.Object is the base class. Hence, the following two pieces of code yield the same result: class MyClass: Object // derives from System.Object { // etc. } and: class MyClass // derives from System.Object { // etc. } For the sake of simplicity, the second form is more common. Because C# supports the object keyword, which serves as a pseudonym for the System.Object class, you can also write this: class MyClass: object // derives from System.Object { // etc. } If you want to reference the Object class, use the object keyword, which is recognized by intelligent editors such as Visual Studio .NET and thus facilitates editing your code. Virtual Methods By declaring a base class function as virtual, you allow the function to be overridden in any derived classes: class MyBaseClass { public virtual string VirtualMethod() { return "This method is virtual and defined in MyBaseClass"; } } c04.indd 91 30-01-2014 20:06:28 92  ❘  CHAPTER 4  Inheritance It is also permitted to declare a property as virtual. For a virtual or overridden property, the syntax is the same as for a nonvirtual property, with the exception of the keyword virtual, which is added to the definition. The syntax looks like this: public virtual string ForeName { get { return foreName;} set { foreName = value;} } private string foreName; For simplicity, the following discussion focuses mainly on methods, but it applies equally well to properties. The concepts behind virtual functions in C# are identical to standard OOP concepts. You can override a virtual function in a derived class; and when the method is called, the appropriate method for the type of object is invoked. In C#, functions are not virtual by default but (aside from constructors) can be explicitly declared as virtual. This follows the C++ methodology: For performance reasons, functions are not virtual unless indicated. In Java, by contrast, all functions are virtual. C# does differ from C++ syntax, though, because it requires you to declare when a derived class’s function overrides another function, using the override keyword: class MyDerivedClass: MyBaseClass { public override string VirtualMethod() { return "This method is an override defined in MyDerivedClass."; } } This syntax for method overriding removes potential runtime bugs that can easily occur in C++, when a method signature in a derived class unintentionally differs slightly from the base version, resulting in the method failing to override the base version. In C#, this is picked up as a compile-time error because the compiler would see a function marked as override but no base method for it to override. Neither member fields nor static functions can be declared as virtual. The concept simply wouldn’t make sense for any class member other than an instance function member. Hiding Methods If a method with the same signature is declared in both base and derived classes but the methods are not declared as virtual and override, respectively, then the derived class version is said to hide the base class version. In most cases, you would want to override methods rather than hide them. By hiding them you risk calling the wrong method for a given class instance. However, as shown in the following example, C# syntax is designed to ensure that the developer is warned at compile time about this potential problem, thus making it safer to hide methods if that is your intention. This also has versioning benefits for developers of class libraries. Suppose that you have a class called HisBaseClass: class HisBaseClass { // various members } At some point in the future, you write a derived class that adds some functionality to HisBaseClass. In particular, you add a method called MyGroovyMethod(), which is not present in the base class: class MyDerivedClass: HisBaseClass { public int MyGroovyMethod() { // some groovy implementation return 0; } } c04.indd 92 30-01-2014 20:06:28 Implementation Inheritance  ❘  93 One year later, you decide to extend the functionality of the base class. By coincidence, you add a method that is also called MyGroovyMethod() and that has the same name and signature as yours, but probably doesn’t do the same thing. When you compile your code using the new version of the base class, you have a potential clash because your program won’t know which method to call. It’s all perfectly legal in C#, but because your MyGroovyMethod() is not intended to be related in any way to the base class MyGroovyMethod(), the result is that running this code does not yield the result you want. Fortunately, C# has been designed to cope very well with these types of conflicts. In these situations, C# generates a compilation warning that reminds you to use the new keyword to declare that you intend to hide a method, like this: class MyDerivedClass: HisBaseClass { public new int MyGroovyMethod() { // some groovy implementation return 0; } } However, because your version of MyGroovyMethod() is not declared as new, the compiler picks up on the fact that it’s hiding a base class method without being instructed to do so and generates a warning (this applies whether or not you declared MyGroovyMethod() as virtual). If you want, you can rename your version of the method. This is the recommended course of action because it eliminates future confusion. However, if you decide not to rename your method for whatever reason (for example, if you’ve published your software as a library for other companies, so you can’t change the names of methods), all your existing client code will still run correctly, picking up your version of MyGroovyMethod(). This is because any existing code that accesses this method must be done through a reference to MyDerivedClass (or a further derived class). Your existing code cannot access this method through a reference to HisBaseClass; it would generate a compilation error when compiled against the earlier version of HisBaseClass. The problem can only occur in client code you have yet to write. C# is designed to issue a warning that a potential problem might occur in future code — you need to pay attention to this warning and take care not to attempt to call your version of MyGroovyMethod() through any reference to HisBaseClass in any future code you add. However, all your existing code will still work fine. It may be a subtle point, but it’s an impressive example of how C# is able to cope with different versions of classes. Calling Base Versions of Functions C# has a special syntax for calling base versions of a method from a derived class: base.(). For example, if you want a method in a derived class to return 90 percent of the value returned by the base class method, you can use the following syntax: class CustomerAccount { public virtual decimal CalculatePrice() { // implementation return 0.0M; } } class GoldAccount: CustomerAccount { public override decimal CalculatePrice() { return base.CalculatePrice() * 0.9M; } } Note that you can use the base.() syntax to call any method in the base class — you don’t have to call it from inside an override of the same method. c04.indd 93 30-01-2014 20:06:28 94  ❘  CHAPTER 4  Inheritance Abstract Classes and Functions C# allows both classes and functions to be declared as abstract. An abstract class cannot be instantiated, whereas an abstract function does not have an implementation, and must be overridden in any non-abstract derived class. Obviously, an abstract function is automatically virtual (although you don’t need to supply the virtual keyword; and doing so results in a syntax error). If any class contains any abstract functions, that class is also abstract and must be declared as such: abstract class Building { public abstract decimal CalculateHeatingCost(); // abstract method } Note  C++ developers should note the slightly different terminology. In C++, abstract functions are often described as pure virtual; in the C# world, the only correct term to use is abstract. Sealed Classes and Methods C# allows classes and methods to be declared as sealed. In the case of a class, this means you can’t inherit from that class. In the case of a method, this means you can’t override that method. sealed class FinalClass { // etc } class DerivedClass: FinalClass // wrong. Will give compilation error { // etc } The most likely situation in which you’ll mark a class or method as sealed is if the class or method is internal to the operation of the library, class, or other classes that you are writing, to ensure that any attempt to override some of its functionality will lead to instability in the code. You might also mark a class or method as sealed for commercial reasons, in order to prevent a third party from extending your classes in a manner that is contrary to the licensing agreements. In general, however, be careful about marking a class or member as sealed, because by doing so you are severely restricting how it can be used. Even if you don’t think it would be useful to inherit from a class or override a particular member of it, it’s still possible that at some point in the future someone will encounter a situation you hadn’t anticipated in which it is useful to do so. The .NET base class library frequently uses sealed classes to make these classes inaccessible to third-party developers who might want to derive their own classes from them. For example, string is a sealed class. Declaring a method as sealed serves a purpose similar to that for a class: class MyClass: MyClassBase { public sealed override void FinalMethod() { // etc. } } class DerivedClass: MyClass { public override void FinalMethod() // wrong. Will give compilation error { } } c04.indd 94 30-01-2014 20:06:28 Implementation Inheritance  ❘  95 In order to use the sealed keyword on a method or property, it must have first been overridden from a base class. If you do not want a method or property in a base class overridden, then don’t mark it as virtual. Constructors of Derived Classes Chapter 3 discusses how constructors can be applied to individual classes. An interesting question arises as to what happens when you start defining your own constructors for classes that are part of a hierarchy, inherited from other classes that may also have custom constructors. Assume that you have not defined any explicit constructors for any of your classes. This means that the compiler supplies default zeroing-out constructors for all your classes. There is actually quite a lot going on under the hood when that happens, but the compiler is able to arrange it so that things work out nicely throughout the class hierarchy and every field in every class is initialized to whatever its default value is. When you add a constructor of your own, however, you are effectively taking control of construction. This has implications right down through the hierarchy of derived classes, so you have to ensure that you don’t inadvertently do anything to prevent construction through the hierarchy from taking place smoothly. You might be wondering why there is any special problem with derived classes. The reason is that when you create an instance of a derived class, more than one constructor is at work. The constructor of the class you instantiate isn’t by itself sufficient to initialize the class — the constructors of the base classes must also be called. That’s why we’ve been talking about construction through the hierarchy. To understand why base class constructors must be called, you’re going to develop an example based on a cell phone company called MortimerPhones. The example contains an abstract base class, GenericCustomer, which represents any customer. There is also a (non-abstract) class, Nevermore60Customer, which represents any customer on a particular rate called the Nevermore60 rate. All customers have a name, represented by a private field. Under the Nevermore60 rate, the first few minutes of the customer’s call time are charged at a higher rate, necessitating the need for the field highCostMinutesUsed, which details how many of these higher-cost minutes each customer has used. The class definitions look like this: abstract class GenericCustomer { private string name; // lots of other methods etc. } class Nevermore60Customer: GenericCustomer { private uint highCostMinutesUsed; // other methods etc. } Don’t worry about what other methods might be implemented in these classes because we are concentrating solely on the construction process here. If you download the sample code for this chapter, you’ll find that the class definitions include only the constructors. Take a look at what happens when you use the new operator to instantiate a Nevermore60Customer: GenericCustomer customer = new Nevermore60Customer(); Clearly, both of the member fields name and highCostMinutesUsed must be initialized when customer is instantiated. If you don’t supply constructors of your own, but rely simply on the default constructors, then you’d expect name to be initialized to the null reference, and highCostMinutesUsed initialized to zero. Let’s look in a bit more detail at how this actually happens. The highCostMinutesUsed field presents no problem: The default Nevermore60Customer constructor supplied by the compiler initializes this field to zero. What about name? Looking at the class definitions, it’s clear that the Nevermore60Customer constructor can’t initialize this value. This field is declared as private, which means that derived classes don’t have access c04.indd 95 30-01-2014 20:06:29 96  ❘  CHAPTER 4  Inheritance to it. Therefore, the default Nevermore60Customer constructor won’t know that this field exists. The only code items that have that knowledge are other members of GenericCustomer. Therefore, if name is going to be initialized, that must be done by a constructor in GenericCustomer. No matter how big your class hierarchy is, this same reasoning applies right down to the ultimate base class, System.Object. Now that you have an understanding of the issues involved, you can look at what actually happens whenever a derived class is instantiated. Assuming that default constructors are used throughout, the compiler first grabs the constructor of the class it is trying to instantiate, in this case Nevermore60Customer. The first thing that the default Nevermore60Customer constructor does is attempt to run the default constructor for the immediate base class, GenericCustomer. The GenericCustomer constructor attempts to run the constructor for its immediate base class, System.Object; but System.Object doesn’t have any base classes, so its constructor just executes and returns control to the GenericCustomer constructor. That constructor now executes, initializing name to null, before returning control to the Nevermore60Customer constructor. That constructor in turn executes, initializing highCostMinutesUsed to zero, and exits. At this point, the Nevermore60Customer instance has been successfully constructed and initialized. The net result of all this is that the constructors are called in order of System.Object first, and then progress down the hierarchy until the compiler reaches the class being instantiated. Notice that in this process, each constructor handles initialization of the fields in its own class. That’s how it should normally work, and when you start adding your own constructors you should try to stick to that principle. Note the order in which this happens. It’s always the base class constructors that are called first. This means there are no problems with a constructor for a derived class invoking any base class methods, properties, and any other members to which it has access, because it can be confident that the base class has already been constructed and its fields initialized. It also means that if the derived class doesn’t like the way that the base class has been initialized, it can change the initial values of the data, provided that it has access to do so. However, good programming practice almost invariably means you’ll try to prevent that situation from occurring if possible, and you will trust the base class constructor to deal with its own fields. Now that you know how the process of construction works, you can start fiddling with it by adding your own constructors. Adding a Constructor in a Hierarchy This section takes the easiest case first and demonstrates what happens if you simply replace the default constructor somewhere in the hierarchy with another constructor that takes no parameters. Suppose that you decide that you want everyone’s name to be initially set to the string "" instead of to the null reference. You’d modify the code in GenericCustomer like this: public abstract class GenericCustomer { private string name; public GenericCustomer() : base() // We could omit this line without affecting the compiled code. { name = ""; } Adding this code will work fine. Nevermore60Customer still has its default constructor, so the sequence of events described earlier will proceed as before, except that the compiler uses the custom GenericCustomer constructor instead of generating a default one, so the name field is always initialized to "" as required. Notice that in your constructor you’ve added a call to the base class constructor before the GenericCustomer constructor is executed, using a syntax similar to that used earlier when you saw how to get different overloads of constructors to call each other. The only difference is that this time you use the base keyword instead of this to indicate that it’s a constructor to the base class, rather than a constructor to the current class, you want to call. There are no parameters in the brackets after the base keyword — that’s important because it means you are not passing any parameters to the base constructor, c04.indd 96 30-01-2014 20:06:29 Implementation Inheritance  ❘  97 so the compiler has to look for a parameterless constructor to call. The result of all this is that the compiler injects code to call the System.Object constructor, which is what happens by default anyway. In fact, you can omit that line of code and write the following (as was done for most of the constructors so far in this chapter): public GenericCustomer() { name = ""; } If the compiler doesn’t see any reference to another constructor before the opening curly brace, it assumes that you wanted to call the base class constructor; this is consistent with how default constructors work. The base and this keywords are the only keywords allowed in the line that calls another constructor. Anything else causes a compilation error. Also note that only one other constructor can be specified. So far, this code works fine. One way to collapse the progression through the hierarchy of constructors, however, is to declare a constructor as private: private GenericCustomer() { name = ""; } If you try this, you’ll get an interesting compilation error, which could really throw you if you don’t understand how construction down a hierarchy works: 'Wrox.ProCSharp.GenericCustomer.GenericCustomer()' is inaccessible due to its protection level What’s interesting here is that the error occurs not in the GenericCustomer class but in the derived class, Nevermore60Customer. That’s because the compiler tried to generate a default constructor for Nevermore60Customer but was not able to, as the default constructor is supposed to invoke the no-parameter GenericCustomer constructor. By declaring that constructor as private, you’ve made it inaccessible to the derived class. A similar error occurs if you supply a constructor to GenericCustomer, which takes parameters, but at the same time you fail to supply a no-parameter constructor. In this case, the compiler won’t generate a default constructor for GenericCustomer, so when it tries to generate the default constructors for any derived class, it again finds that it can’t because a no-parameter base class constructor is not available. A workaround is to add your own constructors to the derived classes — even if you don’t actually need to do anything in these constructors — so that the compiler doesn’t try to generate any default constructors. Now that you have all the theoretical background you need, you’re ready to move on to an example demonstrating how you can neatly add constructors to a hierarchy of classes. In the next section, you start adding constructors that take parameters to the MortimerPhones example. Adding Constructors with Parameters to a Hierarchy You’re going to start with a one-parameter constructor for GenericCustomer, which specifies that customers can be instantiated only when they supply their names: abstract class GenericCustomer { private string name; public GenericCustomer(string name) { this.name = name; } So far, so good. However, as mentioned previously, this causes a compilation error when the compiler tries to create a default constructor for any derived classes because the default compiler-generated constructors for Nevermore60Customer will try to call a no-parameter GenericCustomer constructor, c04.indd 97 30-01-2014 20:06:29 98  ❘  CHAPTER 4  Inheritance and GenericCustomer does not possess such a constructor. Therefore, you need to supply your own constructors to the derived classes to avoid a compilation error: class Nevermore60Customer: GenericCustomer { private uint highCostMinutesUsed; public Nevermore60Customer(string name) : base(name) { } Now instantiation of Nevermore60Customer objects can occur only when a string containing the customer’s name is supplied, which is what you want anyway. The interesting thing here is what the Nevermore60Customer constructor does with this string. Remember that it can’t initialize the name field itself because it has no access to private fields in its base class. Instead, it passes the name through to the base class for the GenericCustomer constructor to handle. It does this by specifying that the base class constructor to be executed first is the one that takes the name as a parameter. Other than that, it doesn’t take any action of its own. Now examine what happens if you have different overloads of the constructor as well as a class hierarchy to deal with. To this end, assume that Nevermore60 customers might have been referred to MortimerPhones by a friend as part of one of those sign-up-a-friend-and-get-a-discount offers. This means that when you construct a Nevermore60Customer, you may need to pass in the referrer’s name as well. In real life, the constructor would have to do something complicated with the name, such as process the discount, but here you’ll just store the referrer’s name in another field. The Nevermore60Customer definition will now look like this: class Nevermore60Customer: GenericCustomer { public Nevermore60Customer(string name, string referrerName) : base(name) { this.referrerName = referrerName; } private string referrerName; private uint highCostMinutesUsed; The constructor takes the name and passes it to the GenericCustomer constructor for processing; referrerName is the variable that is your responsibility here, so the constructor deals with that parameter in its main body. However, not all Nevermore60Customers will have a referrer, so you still need a constructor that doesn’t require this parameter (or a constructor that gives you a default value for it). In fact, you will specify that if there is no referrer, then the referrerName field should be set to "", using the following one- parameter constructor: public Nevermore60Customer(string name) : this(name, "") { } You now have all your constructors set up correctly. It’s instructive to examine the chain of events that occurs when you execute a line like this: GenericCustomer customer = new Nevermore60Customer("Arabel Jones"); The compiler sees that it needs a one-parameter constructor that takes one string, so the constructor it identifies is the last one that you defined: public Nevermore60Customer(string Name) : this(Name, "") c04.indd 98 30-01-2014 20:06:29 Modifiers  ❘  99 When you instantiate customer, this constructor is called. It immediately transfers control to the corresponding Nevermore60Customer two-parameter constructor, passing it the values "ArabelJones", and "". Looking at the code for this constructor, you see that it in turn immediately passes control to the one-parameter GenericCustomer constructor, giving it the string "ArabelJones", and in turn that constructor passes control to the System.Object default constructor. Only now do the constructors execute. First, the System.Object constructor executes. Next is the GenericCustomer constructor, which initializes the name field. Then the Nevermore60Customer two-parameter constructor gets control back and sorts out initializing the referrerName to "". Finally, the Nevermore60Customer one-parameter constructor executes; this constructor doesn’t do anything else. As you can see, this is a very neat and well-designed process. Each constructor handles initialization of the variables that are obviously its responsibility; and, in the process, your class is correctly instantiated and prepared for use. If you follow the same principles when you write your own constructors for your classes, even the most complex classes should be initialized smoothly and without any problems. Modifiers You have already encountered quite a number of so-called modifiers — keywords that can be applied to a type or a member. Modifiers can indicate the visibility of a method, such as public or private, or the nature of an item, such as whether a method is virtual or abstract. C# has a number of modifiers, and at this point it’s worth taking a minute to provide the complete list. Visibility Modifiers Visibility modifiers indicate which other code items can view an item. MODIFIER APPLIES TO DESCRIPTION public Any types or members The item is visible to any other code. protected Any member of a type, and any nested type The item is visible only to any derived type. internal Any types or members The item is visible only within its containing assembly. private Any member of a type, and any nested type The item is visible only inside the type to which it belongs. protected internal Any member of a type, and any nested type The item is visible to any code within its containing assembly and to any code inside a derived type. Note that type definitions can be internal or public, depending on whether you want the type to be visible outside its containing assembly: public class MyClass { // etc. You cannot define types as protected, private, or protected internal because these visibility levels would be meaningless for a type contained in a namespace. Hence, these visibilities can be applied only to members. However, you can define nested types (that is, types contained within other types) with these visibilities because in this case the type also has the status of a member. Hence, the following code is correct: public class OuterClass { protected class InnerClass { // etc. } // etc. } c04.indd 99 30-01-2014 20:06:29 100  ❘  CHAPTER 4  Inheritance If you have a nested type, the inner type is always able to see all members of the outer type. Therefore, with the preceding code, any code inside InnerClass always has access to all members of OuterClass, even where those members are private. Other Modifiers The modifiers in the following table can be applied to members of types and have various uses. A few of these modifiers also make sense when applied to types. MODIFIER APPLIES TO DESCRIPTION new Function members The member hides an inherited member with the same signature. static All members The member does not operate on a specific instance of the class. virtual Function members only The member can be overridden by a derived class. abstract Function members only A virtual member that defines the signature of the member but doesn’t provide an implementation. override Function members only The member overrides an inherited virtual or abstract member. sealed Classes, methods, and properties For classes, the class cannot be inherited from. For properties and methods, the member overrides an inherited virtual member but cannot be overridden by any members in any derived classes. Must be used in conjunction with override. extern Static [DllImport] methods only The member is implemented externally, in a different language. Interfaces As mentioned earlier, by deriving from an interface, a class is declaring that it implements certain functions. Because not all object-oriented languages support interfaces, this section examines C#’s implementation of interfaces in detail. It illustrates interfaces by presenting the complete definition of one of the interfaces that has been predefined by Microsoft — System.IDisposable. IDisposable contains one method, Dispose(), which is intended to be implemented by classes to clean up code: public interface IDisposable { void Dispose(); } This code shows that declaring an interface works syntactically in much the same way as declaring an abstract class. Be aware, however, that it is not permitted to supply implementations of any of the members of an interface. In general, an interface can contain only declarations of methods, properties, indexers, and events. You can never instantiate an interface; it contains only the signatures of its members. An interface has neither constructors (how can you construct something that you can’t instantiate?) nor fields (because that would imply some internal implementation). Nor is an interface definition allowed to contain operator overloads, although that’s not because there is any problem with declaring them; there isn’t, but because interfaces are usually intended to be public contracts, having operator overloads would cause some incompatibility problems with other .NET languages, such as Visual Basic .NET, which do not support operator overloading. Nor is it permitted to declare modifiers on the members in an interface definition. Interface members are always implicitly public, and they cannot be declared as virtual or static. That’s up to implementing c04.indd 100 30-01-2014 20:06:29 Interfaces  ❘  101 classes to decide. Therefore, it is fine for implementing classes to declare access modifiers, as demonstrated in the example in this section. For example, consider IDisposable. If a class wants to declare publicly that it implements the Dispose() method, it must implement IDisposable, which in C# terms means that the class derives from IDisposable: class SomeClass: IDisposable { // This class MUST contain an implementation of the // IDisposable.Dispose() method, otherwise // you get a compilation error. public void Dispose() { // implementation of Dispose() method } // rest of class } In this example, if SomeClass derives from IDisposable but doesn’t contain a Dispose() implementation with the exact same signature as defined in IDisposable, you get a compilation error because the class is breaking its agreed-on contract to implement IDisposable. Of course, it’s no problem for the compiler if a class has a Dispose() method but doesn’t derive from IDisposable. The problem is that other code would have no way of recognizing that SomeClass has agreed to support the IDisposable features. Note  IDisposable is a relatively simple interface because it defines only one method. Most interfaces contain more members. Defining and Implementing Interfaces This section illustrates how to define and use interfaces by developing a short program that follows the interface inheritance paradigm. The example is based on bank accounts. Assume that you are writing code that will ultimately allow computerized transfers between bank accounts. Assume also for this example that there are many companies that implement bank accounts but they have all mutually agreed that any classes representing bank accounts will implement an interface, IBankAccount, which exposes methods to deposit or withdraw money, and a property to return the balance. It is this interface that enables outside code to recognize the various bank account classes implemented by different bank accounts. Although the aim is to enable the bank accounts to communicate with each other to allow transfers of funds between accounts, that feature isn’t introduced just yet. To keep things simple, you will keep all the code for the example in the same source file. Of course, if something like the example were used in real life, you could surmise that the different bank account classes would not only be compiled to different assemblies, but also be hosted on different machines owned by the different banks. That’s all much too complicated for our purposes here. However, to maintain some realism, you will define different namespaces for the different companies. To begin, you need to define the IBankAccount interface: namespace Wrox.ProCSharp { public interface IBankAccount { void PayIn(decimal amount); bool Withdraw(decimal amount); decimal Balance { get; } } } c04.indd 101 30-01-2014 20:06:30 102  ❘  CHAPTER 4  Inheritance Notice the name of the interface, IBankAccount. It’s a best-practice convention to begin an interface name with the letter I, to indicate it’s an interface. Note  Chapter 2, “Core C#,” points out that in most cases, .NET usage guidelines discourage the so-called Hungarian notation in which names are preceded by a letter that indicates the type of object being defined. Interfaces are one of the few exceptions for which Hungarian notation is recommended. The idea is that you can now write classes that represent bank accounts. These classes don’t have to be related to each other in any way; they can be completely different classes. They will all, however, declare that they represent bank accounts by the mere fact that they implement the IBankAccount interface. Let’s start off with the first class, a saver account run by the Royal Bank of Venus: namespace Wrox.ProCSharp.VenusBank { public class SaverAccount: IBankAccount { private decimal balance; public void PayIn(decimal amount) { balance += amount; } public bool Withdraw(decimal amount) { if (balance >= amount) { balance -= amount; return true; } Console.WriteLine("Withdrawal attempt failed."); return false; } public decimal Balance { get { return balance; } } public override string ToString() { return String.Format("Venus Bank Saver: Balance = {0,6:C}", balance); } } } It should be obvious what the implementation of this class does. You maintain a private field, balance, and adjust this amount when money is deposited or withdrawn. You display an error message if an attempt to withdraw money fails because of insufficient funds. Notice also that because we are keeping the code as simple as possible, we are not implementing extra properties, such as the account holder’s name! In real life that would be essential information, of course, but for this example it’s unnecessarily complicated. The only really interesting line in this code is the class declaration: public class SaverAccount: IBankAccount You’ve declared that SaverAccount is derived from one interface, IBankAccount, and you have not explicitly indicated any other base classes (which of course means that SaverAccount is derived directly from System .Object). By the way, derivation from interfaces acts completely independently from derivation from classes. c04.indd 102 30-01-2014 20:06:30 Interfaces  ❘  103 Being derived from IBankAccount means that SaverAccount gets all the members of IBankAccount; but because an interface doesn’t actually implement any of its methods, SaverAccount must provide its own implementations of all of them. If any implementations are missing, you can rest assured that the compiler will complain. Recall also that the interface just indicates the presence of its members. It’s up to the class to determine whether it wants any of them to be virtual or abstract (though abstract functions are of course only allowed if the class itself is abstract). For this particular example, you don’t have any reason to make any of the interface functions virtual. To illustrate how different classes can implement the same interface, assume that the Planetary Bank of Jupiter also implements a class to represent one of its bank accounts — a Gold Account: namespace Wrox.ProCSharp.JupiterBank { public class GoldAccount: IBankAccount { // etc } } We won’t present details of the GoldAccount class here; in the sample code, it’s basically identical to the implementation of SaverAccount. We stress that GoldAccount has no connection with SaverAccount, other than they both happen to implement the same interface. Now that you have your classes, you can test them. You first need a few using statements: using System; using Wrox.ProCSharp; using Wrox.ProCSharp.VenusBank; using Wrox.ProCSharp.JupiterBank; Now you need a Main() method: namespace Wrox.ProCSharp { class MainEntryPoint { static void Main() { IBankAccount venusAccount = new SaverAccount(); IBankAccount jupiterAccount = new GoldAccount(); venusAccount.PayIn(200); venusAccount.Withdraw(100); Console.WriteLine(venusAccount.ToString()); jupiterAccount.PayIn(500); jupiterAccount.Withdraw(600); jupiterAccount.Withdraw(100); Console.WriteLine(jupiterAccount.ToString()); } } } This code (which if you download the sample, you can find in the file BankAccounts.cs) produces the following output: C:> BankAccounts Venus Bank Saver: Balance = £100.00 Withdrawal attempt failed. Jupiter Bank Saver: Balance = £400.00 The main point to notice about this code is the way that you have declared both your reference variables as IBankAccount references. This means that they can point to any instance of any class that implements this interface. However, it also means that you can call only methods that are part of this interface through these references — if you want to call any methods implemented by a class that are not part of the interface, you need to cast the reference to the appropriate type. In the example code, you were able to call ToString() (not implemented by IBankAccount) without any explicit cast, purely because ToString() is c04.indd 103 30-01-2014 20:06:30 104  ❘  CHAPTER 4  Inheritance a System.Object method, so the C# compiler knows that it will be supported by any class (put differently, the cast from any interface to System.Object is implicit). Chapter 7, “Operators and Casts,” covers the syntax for performing casts. Interface references can in all respects be treated as class references — but the power of an interface reference is that it can refer to any class that implements that interface. For example, this allows you to form arrays of interfaces, whereby each element of the array is a different class: IBankAccount[] accounts = new IBankAccount[2]; accounts[0] = new SaverAccount(); accounts[1] = new GoldAccount(); Note, however, that you would get a compiler error if you tried something like this: accounts[1] = new SomeOtherClass(); // SomeOtherClass does NOT implement // IBankAccount: WRONG!! The preceding causes a compilation error similar to this: Cannot implicitly convert type 'Wrox.ProCSharp. SomeOtherClass' to 'Wrox.ProCSharp.IBankAccount' Derived Interfaces It’s possible for interfaces to inherit from each other in the same way that classes do. This concept is illustrated by defining a new interface, ITransferBankAccount, which has the same features as IBankAccount but also defines a method to transfer money directly to a different account: namespace Wrox.ProCSharp { public interface ITransferBankAccount: IBankAccount { bool TransferTo(IBankAccount destination, decimal amount); } } Because ITransferBankAccount is derived from IBankAccount, it gets all the members of IBankAccount as well as its own. That means that any class that implements (derives from) ITransferBankAccount must implement all the methods of IBankAccount, as well as the new TransferTo() method defined in ITransferBankAccount. Failure to implement all these methods will result in a compilation error. Note that the TransferTo() method uses an IBankAccount interface reference for the destination account. This illustrates the usefulness of interfaces: When implementing and then invoking this method, you don’t need to know anything about what type of object you are transferring money to — all you need to know is that this object implements IBankAccount. To illustrate ITransferBankAccount, assume that the Planetary Bank of Jupiter also offers a current account. Most of the implementation of the CurrentAccount class is identical to implementations of SaverAccount and GoldAccount (again, this is just to keep this example simple — that won’t normally be the case), so in the following code only the differences are highlighted: public class CurrentAccount: ITransferBankAccount { private decimal balance; public void PayIn(decimal amount) { balance += amount; } public bool Withdraw(decimal amount) { if (balance >= amount) { balance -= amount; c04.indd 104 30-01-2014 20:06:30 Summary  ❘  105 return true; } Console.WriteLine("Withdrawal attempt failed."); return false; } public decimal Balance { get { return balance; } } public bool TransferTo(IBankAccount destination, decimal amount) { bool result; result = Withdraw(amount); if (result) { destination.PayIn(amount); } return result; } public override string ToString() { return String.Format("Jupiter Bank Current Account: Balance = {0,6:C}",balance); } } The class can be demonstrated with this code: static void Main() { IBankAccount venusAccount = new SaverAccount(); ITransferBankAccount jupiterAccount = new CurrentAccount(); venusAccount.PayIn(200); jupiterAccount.PayIn(500); jupiterAccount.TransferTo(venusAccount, 100); Console.WriteLine(venusAccount.ToString()); Console.WriteLine(jupiterAccount.ToString()); } The preceding code (CurrentAccounts.cs) produces the following output, which, as you can verify, shows that the correct amounts have been transferred: C:> CurrentAccount Venus Bank Saver: Balance = £300.00 Jupiter Bank Current Account: Balance = £400.00 Summary This chapter described how to code inheritance in C#. You have seen that C# offers rich support for both multiple interface and single implementation inheritance. You have also learned that C# provides a number of useful syntactical constructs designed to assist in making code more robust. These include the override keyword, which indicates when a function should override a base function; the new keyword, which indicates when a function hides a base function; and rigid rules for constructor initializers that are designed to ensure that constructors are designed to interoperate in a robust manner. c04.indd 105 30-01-2014 20:06:30 c04.indd 106 30-01-2014 20:06:30 Generics WHAT’S iN THiS CHAPTER? ➤➤ An overview of generics ➤➤ Creating generic classes ➤➤ Features of generic classes ➤➤ Generic interfaces ➤➤ Generic structs ➤➤ Generic methods WROX.COM CODE DOWNlOADS FOR THiS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ Linked List Objects ➤➤ Linked List Sample ➤➤ Document Manager ➤➤ Variance ➤➤ Generic Methods ➤➤ Specialization GENERiCS OVERViEW Since the release of .NET 2.0, .NET has supported generics. Generics are not just a part of the C# programming language; they are deeply integrated with the IL (Intermediate Language) code in the assemblies. With generics, you can create classes and methods that are independent of contained types. Instead of writing a number of methods or classes with the same functionality for different types, you can create just one method or class. Another option to reduce the amount of code is using the Object class. However, passing using types derived from the Object class is not type safe. Generic classes make use of generic types that are 5 c05.indd 107 30-01-2014 20:11:51 108  ❘  CHAPTER 5  Generics replaced with specific types as needed. This allows for type safety: the compiler complains if a specific type is not supported with the generic class. Generics are not limited to classes; in this chapter, you also see generics with interfaces and methods. Generics with delegates can be found in Chapter 8, “Delegates, Lambdas, and Events.” Generics are not a completely new construct; similar concepts exist with other languages. For example, C++ templates have some similarity to generics. However, there’s a big difference between C++ templates and .NET generics. With C++ templates, the source code of the template is required when a template is instantiated with a specific type. Unlike C++ templates, generics are not only a construct of the C# language, but are defined with the CLR. This makes it possible to instantiate generics with a specific type in Visual Basic even though the generic class was defined with C#. The following sections explore the advantages and disadvantages of generics, particularly in regard to the following: ➤➤ Performance ➤➤ Type safety ➤➤ Binary code reuse ➤➤ Code bloat ➤➤ Naming guidelines Performance One of the big advantages of generics is performance. In Chapter 10, “Collections,” you will see non-generic and generic collection classes from the namespaces System.Collections and System.Collections .Generic. Using value types with non-generic collection classes results in boxing and unboxing when the value type is converted to a reference type, and vice versa. NOTE  Boxing and unboxing are discussed in Chapter 7, “Operators and Casts.” Here is just a short refresher about these terms. Value types are stored on the stack, whereas reference types are stored on the heap. C# classes are reference types; structs are value types. .NET makes it easy to convert value types to reference types, so you can use a value type everywhere an object (which is a reference type) is needed. For example, an int can be assigned to an object. The conversion from a value type to a reference type is known as boxing. Boxing occurs automatically if a method requires an object as a parameter, and a value type is passed. In the other direction, a boxed value type can be converted to a value type by using unboxing. With unboxing, the cast operator is required. The following example shows the ArrayList class from the namespace System.Collections. ArrayList stores objects; the Add() method is defined to require an object as a parameter, so an integer type is boxed. When the values from an ArrayList are read, unboxing occurs when the object is converted to an integer type. This may be obvious with the cast operator that is used to assign the first element of the ArrayList collection to the variable i1, but it also happens inside the foreach statement where the variable i2 of type int is accessed: var list = new ArrayList(); list.Add(44); // boxing — convert a value type to a reference type int i1 = (int)list[0]; // unboxing — convert a reference type to // a value type c05.indd 108 30-01-2014 20:11:51 Generics Overview  ❘  109 foreach (int i2 in list) { Console.WriteLine(i2); // unboxing } Boxing and unboxing are easy to use but have a big performance impact, especially when iterating through many items. Instead of using objects, the List class from the namespace System.Collections.Generic enables you to define the type when it is used. In the example here, the generic type of the List class is defined as int, so the int type is used inside the class that is generated dynamically from the JIT compiler. Boxing and unboxing no longer happen: var list = new List(); list.Add(44); // no boxing — value types are stored in the List int i1 = list[0]; // no unboxing, no cast needed foreach (int i2 in list) { Console.WriteLine(i2); } Type Safety Another feature of generics is type safety. As with the ArrayList class, if objects are used, any type can be added to this collection. The following example shows adding an integer, a string, and an object of type MyClass to the collection of type ArrayList: var list = new ArrayList(); list.Add(44); list.Add("mystring"); list.Add(new MyClass()); If this collection is iterated using the following foreach statement, which iterates using integer elements, the compiler accepts this code. However, because not all elements in the collection can be cast to an int, a runtime exception will occur: foreach (int i in list) { Console.WriteLine(i); } Errors should be detected as early as possible. With the generic class List, the generic type T defines what types are allowed. With a definition of List, only integer types can be added to the collection. The compiler doesn’t compile this code because the Add() method has invalid arguments: var list = new List(); list.Add(44); list.Add("mystring"); // compile time error list.Add(new MyClass()); // compile time error Binary Code Reuse Generics enable better binary code reuse. A generic class can be defined once and can be instantiated with many different types. Unlike C++ templates, it is not necessary to access the source code. c05.indd 109 30-01-2014 20:11:51 110  ❘  CHAPTER 5  Generics For example, here the List class from the namespace System.Collections.Generic is instantiated with an int, a string, and a MyClass type: var list = new List(); list.Add(44); var stringList = new List(); stringList.Add("mystring"); var myClassList = new List(); myClassList.Add(new MyClass()); Generic types can be defined in one language and used from any other .NET language. Code Bloat You might be wondering how much code is created with generics when instantiating them with different specific types. Because a generic class definition goes into the assembly, instantiating generic classes with specific types doesn’t duplicate these classes in the IL code. However, when the generic classes are compiled by the JIT compiler to native code, a new class for every specific value type is created. Reference types share all the same implementation of the same native class. This is because with reference types, only a 4-byte memory address (with 32-bit systems) is needed within the generic instantiated class to reference a reference type. Value types are contained within the memory of the generic instantiated class; and because every value type can have different memory requirements, a new class for every value type is instantiated. Naming Guidelines If generics are used in the program, it helps when generic types can be distinguished from non-generic types. Here are naming guidelines for generic types: ➤➤ Generic type names should be prefixed with the letter T. ➤➤ If the generic type can be replaced by any class because there’s no special requirement, and only one generic type is used, the character T is good as a generic type name: public class List { } public class LinkedList { } ➤➤ If there’s a special requirement for a generic type (for example, it must implement an interface or derive from a base class), or if two or more generic types are used, descriptive names should be used for the type names: public delegate void EventHandler(object sender, TEventArgs e); public delegate TOutput Converter(TInput from); public class SortedList { } Creating Generic Classes The example in this section starts with a normal, non-generic simplified linked list class that can contain objects of any kind, and then converts this class to a generic class. With a linked list, one element references the next one. Therefore, you must create a class that wraps the object inside the linked list and references the next object. The class LinkedListNode contains a property named Value that is initialized with the constructor. In addition to that, the LinkedListNode class contains references to the next and previous elements in the list that can be accessed from properties (code file LinkedListObjects/LinkedListNode.cs): c05.indd 110 30-01-2014 20:11:52 Creating Generic Classes  ❘  111 public class LinkedListNode { public LinkedListNode(object value) { this.Value = value; } public object Value { get; private set; } public LinkedListNode Next { get; internal set; } public LinkedListNode Prev { get; internal set; } } The LinkedList class includes First and Last properties of type LinkedListNode that mark the beginning and end of the list. The method AddLast() adds a new element to the end of the list. First, an object of type LinkedListNode is created. If the list is empty, then the First and Last properties are set to the new element; otherwise, the new element is added as the last element to the list. By implementing the GetEnumerator() method, it is possible to iterate through the list with the foreach statement. The GetEnumerator() method makes use of the yield statement for creating an enumerator type: public class LinkedList: IEnumerable { public LinkedListNode First { get; private set; } public LinkedListNode Last { get; private set; } public LinkedListNode AddLast(object node) { var newNode = new LinkedListNode(node); if (First == null) { First = newNode; Last = First; } else { LinkedListNode previous = Last; Last.Next = newNode; Last = newNode; Last.Prev = previous; } return newNode; } public IEnumerator GetEnumerator() { LinkedListNode current = First; while (current != null) { yield return current.Value; current = current.Next; } } } NOTE  The yield statement creates a state machine for an enumerator. This statement is explained in Chapter 6, “Arrays and Tuples.” c05.indd 111 30-01-2014 20:11:52 112  ❘  CHAPTER 5  Generics Now you can use the LinkedList class with any type. The following code segment instantiates a new LinkedList object and adds two integer types and one string type. As the integer types are converted to an object, boxing occurs as explained earlier. With the foreach statement, unboxing happens. In the foreach statement, the elements from the list are cast to an integer, so a runtime exception occurs with the third element in the list because casting to an int fails (code file LinkedListObjects/Program.cs): var list1 = new LinkedList(); list1.AddLast(2); list1.AddLast(4); list1.AddLast("6"); foreach (int i in list1) { Console.WriteLine(i); } Now let’s make a generic version of the linked list. A generic class is defined similarly to a normal class with the generic type declaration. The generic type can then be used within the class as a field member, or with parameter types of methods. The class LinkedListNode is declared with a generic type T. The property Value is now type T instead of object; the constructor is changed as well to accept an object of type T. A generic type can also be returned and set, so the properties Next and Prev are now of type LinkedListNode (code file LinkedListSample/LinkedListNode.cs): public class LinkedListNode { public LinkedListNode(T value) { this.Value = value; } public T Value { get; private set; } public LinkedListNode Next { get; internal set; } public LinkedListNode Prev { get; internal set; } } In the following code the class LinkedList is changed to a generic class as well. LinkedList contains LinkedListNode elements. The type T from the LinkedList defines the type T of the properties First and Last. The method AddLast() now accepts a parameter of type T and instantiates an object of LinkedListNode. Besides the interface IEnumerable, a generic version is also available: IEnumerable. IEnumerable derives from IEnumerable and adds the GetEnumerator() method, which returns IEnumerator. LinkedList implements the generic interface IEnumerable (code file LinkedListSample/ LinkedList.cs): NOTE  Enumerations and the interfaces IEnumerable and IEnumerator are discussed in Chapter 6. public class LinkedList: IEnumerable { public LinkedListNode First { get; private set; } public LinkedListNode Last { get; private set; } public LinkedListNode AddLast(T node) { c05.indd 112 30-01-2014 20:11:52 Creating Generic Classes  ❘  113 var newNode = new LinkedListNode(node); if (First == null) { First = newNode; Last = First; } else { LinkedListNode previous = Last; Last.Next = newNode; Last = newNode; Last.Prev = previous; } return newNode; } public IEnumerator GetEnumerator() { LinkedListNode current = First; while (current != null) { yield return current.Value; current = current.Next; } } IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); } } Using the generic LinkedList, you can instantiate it with an int type, and there’s no boxing. Also, you get a compiler error if you don’t pass an int with the method AddLast(). Using the generic IEnumerable, the foreach statement is also type safe, and you get a compiler error if that variable in the foreach statement is not an int (code file LinkedListSample/Program.cs): var list2 = new LinkedList(); list2.AddLast(1); list2.AddLast(3); list2.AddLast(5); foreach (int i in list2) { Console.WriteLine(i); } Similarly, you can use the generic LinkedList with a string type and pass strings to the AddLast() method: var list3 = new LinkedList(); list3.AddLast("2"); list3.AddLast("four"); list3.AddLast("foo"); foreach (string s in list3) { Console.WriteLine(s); } c05.indd 113 30-01-2014 20:11:52 114  ❘  CHAPTER 5  Generics NOTE  Every class that deals with the object type is a possible candidate for a generic implementation. Also, if classes make use of hierarchies, generics can be very helpful in making casting unnecessary. Generics Features When creating generic classes, you might need some additional C# keywords. For example, it is not possible to assign null to a generic type. In this case, the keyword default can be used, as demonstrated in the next section. If the generic type does not require the features of the Object class but you need to invoke some specific methods in the generic class, you can define constraints. This section discusses the following topics: ➤➤ Default values ➤➤ Constraints ➤➤ Inheritance ➤➤ Static members This example begins with a generic document manager, which is used to read and write documents from and to a queue. Start by creating a new Console project named DocumentManager and add the class DocumentManager. The method AddDocument() adds a document to the queue. The read-only property IsDocumentAvailable returns true if the queue is not empty (code file DocumentManager/ DocumentManager.cs): using System; using System.Collections.Generic; namespace Wrox.ProCSharp.Generics { public class DocumentManager { private readonly Queue documentQueue = new Queue(); public void AddDocument(T doc) { lock (this) { documentQueue.Enqueue(doc); } } public bool IsDocumentAvailable { get { return documentQueue.Count > 0; } } } } Threading and the lock statement are discussed in Chapter 21, “Threads, Tasks, and Synchronization.” Default Values Now you add a GetDocument() method to the DocumentManager class. Inside this method the type T should be assigned to null. However, it is not possible to assign null to generic types. That’s because a generic type can also be instantiated as a value type, and null is allowed only with reference types. c05.indd 114 30-01-2014 20:11:52 Generics Features  ❘  115 To circumvent this problem, you can use the default keyword. With the default keyword, null is assigned to reference types and 0 is assigned to value types: public T GetDocument() { T doc = default(T); lock (this) { doc = documentQueue.Dequeue(); } return doc; } NOTE  The default keyword has multiple meanings depending on the context, or where it is used. The switch statement uses a default for defining the default case, and with generics default is used to initialize generic types either to null or to 0, depending on if it is a reference or value type. Constraints If the generic class needs to invoke some methods from the generic type, you have to add constraints. With DocumentManager, all the document titles should be displayed in the DisplayAllDocuments() method. The Document class implements the interface IDocument with the properties Title and Content (code file DocumentManager/Document.cs): public interface IDocument { string Title { get; set; } string Content { get; set; } } public class Document: IDocument { public Document() { } public Document(string title, string content) { this.Title = title; this.Content = content; } public string Title { get; set; } public string Content { get; set; } } To display the documents with the DocumentManager class, you can cast the type T to the interface IDocument to display the title (code file DocumentManager/DocumentManager.cs): public void DisplayAllDocuments() { foreach (T doc in documentQueue) { c05.indd 115 30-01-2014 20:11:52 116  ❘  CHAPTER 5  Generics Console.WriteLine(((IDocument)doc).Title); } } The problem here is that doing a cast results in a runtime exception if type T does not implement the interface IDocument. Instead, it would be better to define a constraint with the DocumentManager class specifying that the type TDocument must implement the interface IDocument. To clarify the requirement in the name of the generic type, T is changed to TDocument. The where clause defines the requirement to implement the interface IDocument: public class DocumentManager where TDocument: IDocument { This way you can write the foreach statement in such a way that the type TDocument contains the property Title. You get support from Visual Studio IntelliSense and the compiler: public void DisplayAllDocuments() { foreach (TDocument doc in documentQueue) { Console.WriteLine(doc.Title); } } In the Main() method, the DocumentManager class is instantiated with the type Document that implements the required interface IDocument. Then new documents are added and displayed, and one of the documents is retrieved (code file DocumentManager/Program.cs): static void Main() { var dm = new DocumentManager(); dm.AddDocument(new Document("Title A", "Sample A")); dm.AddDocument(new Document("Title B", "Sample B")); dm.DisplayAllDocuments(); if (dm.IsDocumentAvailable) { Document d = dm.GetDocument(); Console.WriteLine(d.Content); } } The DocumentManager now works with any class that implements the interface IDocument. In the sample application, you’ve seen an interface constraint. Generics support several constraint types, indicated in the following table. Constraint Description where T: struct With a struct constraint, type T must be a value type. where T: class The class constraint indicates that type T must be a reference type. where T: IFoo Specifies that type T is required to implement interface IFoo. where T: Foo Specifies that type T is required to derive from base class Foo. where T: new() A constructor constraint; specifies that type T must have a default constructor. where T1: T2 With constraints it is also possible to specify that type T1 derives from a generic type T2. This constraint is known as naked type constraint. c05.indd 116 30-01-2014 20:11:53 Generics Features  ❘  117 NOTE  Constructor constraints can be defined only for the default constructor. It is not possible to define a constructor constraint for other constructors. With a generic type, you can also combine multiple constraints. The constraint where T: IFoo, new() with the MyClass declaration specifies that type T implements the interface IFoo and has a default constructor: public class MyClass where T: IFoo, new() { //... NOTE  One important restriction of the where clause with C# is that it’s not possible to define operators that must be implemented by the generic type. Operators cannot be defined in interfaces. With the where clause, it is only possible to define base classes, interfaces, and the default constructor. Inheritance The LinkedList class created earlier implements the interface IEnumerable: public class LinkedList: IEnumerable { //... A generic type can implement a generic interface. The same is possible by deriving from a class. A generic class can be derived from a generic base class: public class Base { } public class Derived: Base { } The requirement is that the generic types of the interface must be repeated, or the type of the base class must be specified, as in this case: public class Base { } public class Derived: Base { } This way, the derived class can be a generic or non-generic class. For example, you can define an abstract generic base class that is implemented with a concrete type in the derived class. This enables you to write generic specialization for specific types: public abstract class Calc { public abstract T Add(T x, T y); c05.indd 117 30-01-2014 20:11:53 118  ❘  CHAPTER 5  Generics public abstract T Sub(T x, T y); } public class IntCalc: Calc { public override int Add(int x, int y) { return x + y; } public override int Sub(int x, int y) { return x — y; } } Static Members Static members of generic classes are only shared with one instantiation of the class, and require special attention. Consider the following example, where the class StaticDemo contains the static field x: public class StaticDemo { public static int x; } Because the class StaticDemo is used with both a string type and an int type, two sets of static fields exist: StaticDemo.x = 4; StaticDemo.x = 5; Console.WriteLine(StaticDemo.x); // writes 4 Generic Interfaces Using generics, you can define interfaces that define methods with generic parameters. In the linked list sample, you’ve already implemented the interface IEnumerable, which defines a GetEnumerator() method to return IEnumerator. .NET offers a lot of generic interfaces for different scenarios; examples include IComparable, ICollection, and IExtensibleObject. Often older, non- generic versions of the same interface exist; for example .NET 1.0 had an IComparable interface that was based on objects. IComparable is based on a generic type: public interface IComparable { int CompareTo(T other); } The older, non-generic IComparable interface requires an object with the CompareTo() method. This requires a cast to specific types, such as to the Person class for using the LastName property: public class Person: IComparable { public int CompareTo(object obj) { Person other = obj as Person; return this.lastname.CompareTo(other.LastName); } // c05.indd 118 30-01-2014 20:11:53 Generic Interfaces  ❘  119 When implementing the generic version, it is no longer necessary to cast the object to a Person: public class Person: IComparable { public int CompareTo(Person other) { return this.LastName.CompareTo(other.LastName); } //... Covariance and Contra-variance Prior to .NET 4, generic interfaces were invariant. .NET 4 added important changes for generic interfaces and generic delegates: covariance and contra-variance. Covariance and contra-variance are used for the conversion of types with arguments and return types. For example, can you pass a Rectangle to a method that requests a Shape? Let’s get into examples to see the advantages of these extensions. With .NET, parameter types are covariant. Assume you have the classes Shape and Rectangle, and Rectangle derives from the Shape base class. The Display() method is declared to accept an object of the Shape type as its parameter: public void Display(Shape o) { } Now you can pass any object that derives from the Shape base class. Because Rectangle derives from Shape, a Rectangle fulfills all the requirements of a Shape and the compiler accepts this method call: var r = new Rectangle { Width= 5, Height=2.5 }; Display(r); Return types of methods are contra-variant. When a method returns a Shape it is not possible to assign it to a Rectangle because a Shape is not necessarily always a Rectangle; but the opposite is possible. If a method returns a Rectangle as the GetRectangle() method, public Rectangle GetRectangle(); the result can be assigned to a Shape: Shape s = GetRectangle(); Before version 4 of the .NET Framework, this behavior was not possible with generics. Since C# 4, the language is extended to support covariance and contra-variance with generic interfaces and generic delegates. Let’s start by defining a Shape base class and a Rectangle class (code files Variance/Shape.cs and Rectangle.cs): public class Shape { public double Width { get; set; } public double Height { get; set; } public override string ToString() { return String.Format("Width: {0}, Height: {1}", Width, Height); } } public class Rectangle: Shape { } c05.indd 119 30-01-2014 20:11:53 120  ❘  CHAPTER 5  Generics Covariance with Generic Interfaces A generic interface is covariant if the generic type is annotated with the out keyword. This also means that type T is allowed only with return types. The interface IIndex is covariant with type T and returns this type from a read-only indexer (code file Variance/IIndex.cs): public interface IIndex { T this[int index] { get; } int Count { get; } } The IIndex interface is implemented with the RectangleCollection class. RectangleCollection defines Rectangle for generic type T: NOTE  If a read-write indexer is used with the IIndex interface, the generic type T is passed to the method and retrieved from the method. This is not possible with covariance; the generic type must be defined as invariant. Defining the type as invariant is done without out and in annotations (code file Variance/ RectangleCollection.cs): public class RectangleCollection: IIndex { private Rectangle[] data = new Rectangle[3] { new Rectangle { Height=2, Width=5 }, new Rectangle { Height=3, Width=7 }, new Rectangle { Height=4.5, Width=2.9 } }; private static RectangleCollection coll; public static RectangleCollection GetRectangles() { return coll ?? (coll = new RectangleCollection()); } public Rectangle this[int index] { get { if (index < 0 || index > data.Length) throw new ArgumentOutOfRangeException("index"); return data[index]; } } public int Count { get { return data.Length; } } } c05.indd 120 30-01-2014 20:11:53 Generic Interfaces  ❘  121 NOTE  The RectangleCollection.GetRectangles() method makes use of the coalescing operator that is, explained later in this chapter. If the variable coll is null, the right side of operator is invoked to create a new instance of RectangleCollection and assign it to the variable coll, which is returned from this method afterwards. The RectangleCollection.GetRectangles() method returns a RectangleCollection that implements the IIndex interface, so you can assign the return value to a variable rectangle of the IIndex type. Because the interface is covariant, it is also possible to assign the returned value to a variable of IIndex. Shape does not need anything more than a Rectangle has to offer. Using the shapes variable, the indexer from the interface and the Count property are used within the for loop (code file Variance/Program.cs): static void Main() { IIndex rectangles = RectangleCollection.GetRectangles(); IIndex shapes = rectangles; for (int i = 0; i < shapes.Count; i++) { Console.WriteLine(shapes[i]); } } Contra-Variance with Generic Interfaces A generic interface is contra-variant if the generic type is annotated with the in keyword. This way, the interface is only allowed to use generic type T as input to its methods (code file Variance/IDisplay.cs): public interface IDisplay { void Show(T item); } The ShapeDisplay class implements IDisplay and uses a Shape object as an input parameter (code file Variance/ShapeDisplay.cs): public class ShapeDisplay: IDisplay { public void Show(Shape s) { Console.WriteLine("{0} Width: {1}, Height: {2}", s.GetType().Name, s.Width, s.Height); } } Creating a new instance of ShapeDisplay returns IDisplay, which is assigned to the shapeDisplay variable. Because IDisplay is contra-variant, it is possible to assign the result to IDisplay, where Rectangle derives from Shape. This time the methods of the interface define only the generic type as input, and Rectangle fulfills all the requirements of a Shape (code file Variance/Program.cs): static void Main() { //... IDisplay shapeDisplay = new ShapeDisplay(); c05.indd 121 30-01-2014 20:11:53 122  ❘  CHAPTER 5  Generics IDisplay rectangleDisplay = shapeDisplay; rectangleDisplay.Show(rectangles[0]); } Generic Structs Similar to classes, structs can be generic as well. They are very similar to generic classes with the exception of inheritance features. In this section you look at the generic struct Nullable, which is defined by the .NET Framework. An example of a generic struct in the .NET Framework is Nullable. A number in a database and a number in a programming language have an important difference: A number in the database can be null, whereas a number in C# cannot be null. Int32 is a struct, and because structs are implemented as value types, they cannot be null. This difference often causes headaches and a lot of additional work to map the data. The problem exists not only with databases but also with mapping XML data to .NET types. One solution is to map numbers from databases and XML files to reference types, because reference types can have a null value. However, this also means additional overhead during runtime. With the structure Nullable, this can be easily resolved. The following code segment shows a simplified version of how Nullable is defined. The structure Nullable defines a constraint specifying that the generic type T needs to be a struct. With classes as generic types, the advantage of low overhead is eliminated; and because objects of classes can be null anyway, there’s no point in using a class with the Nullable type. The only overhead in addition to the T type defined by Nullable is the hasValue Boolean field that defines whether the value is set or null. Other than that, the generic struct defines the read-only properties HasValue and Value and some operator overloads. The operator overload to cast the Nullable type to T is defined as explicit because it can throw an exception in case hasValue is false. The operator overload to cast to Nullable is defined as implicit because it always succeeds: public struct Nullable where T: struct { public Nullable(T value) { this.hasValue = true; this.value = value; } private bool hasValue; public bool HasValue { get { return hasValue; } } private T value; public T Value { get { if (!hasValue) { throw new InvalidOperationException("no value"); } return value; } } c05.indd 122 30-01-2014 20:11:54 Generic Structs  ❘  123 public static explicit operator T(Nullable value) { return value.Value; } public static implicit operator Nullable(T value) { return new Nullable(value); } public override string ToString() { if (!HasValue) return String.Empty; return this.value.ToString(); } } In this example, Nullable is instantiated with Nullable. The variable x can now be used as an int, assigning values and using operators to do some calculation. This behavior is made possible by casting operators of the Nullable type. However, x can also be null. The Nullable properties HasValue and Value can check whether there is a value, and the value can be accessed: Nullable x; x = 4; x += 3; if (x.HasValue) { int y = x.Value; } x = null; Because nullable types are used often, C# has a special syntax for defining variables of this type. Instead of using syntax with the generic structure, the ? operator can be used. In the following example, the variables x1 and x2 are both instances of a nullable int type: Nullable x1; int? x2; A nullable type can be compared with null and numbers, as shown. Here, the value of x is compared with null, and if it is not null it is compared with a value less than 0: int? x = GetNullableType(); if (x == null) { Console.WriteLine("x is null"); } else if (x < 0) { Console.WriteLine("x is smaller than 0"); } Now that you know how Nullable is defined, let’s get into using nullable types. Nullable types can also be used with arithmetic operators. The variable x3 is the sum of the variables x1 and x2. If any of the nullable types have a null value, the result is null: int? x1 = GetNullableType(); int? x2 = GetNullableType(); int? x3 = x1 + x2; c05.indd 123 30-01-2014 20:11:54 124  ❘  CHAPTER 5  Generics NOTE  The GetNullableType()method, which is called here, is just a placeholder for any method that returns a nullable int. For testing you can implement it to simply return null or to return any integer value. Non-nullable types can be converted to nullable types. With the conversion from a non-nullable type to a nullable type, an implicit conversion is possible where casting is not required. This type of conversion always succeeds: int y1 = 4; int? x1 = y1; In the reverse situation, a conversion from a nullable type to a non-nullable type can fail. If the nullable type has a null value and the null value is assigned to a non-nullable type, then an exception of type InvalidOperationException is thrown. That’s why the cast operator is required to do an explicit conversion: int? x1 = GetNullableType(); int y1 = (int)x1; Instead of doing an explicit cast, it is also possible to convert a nullable type to a non-nullable type with the coalescing operator. The coalescing operator uses the syntax ?? to define a default value for the conversion in case the nullable type has a value of null. Here, y1 gets a 0 value if x1 is null: int? x1 = GetNullableType(); int y1 = x1 ?? 0; Generic Methods In addition to defining generic classes, it is also possible to define generic methods. With a generic method, the generic type is defined with the method declaration. Generic methods can be defined within non-generic classes. The method Swap() defines T as a generic type that is used for two arguments and a variable temp: void Swap(ref T x, ref T y) { T temp; temp = x; x = y; y = temp; } A generic method can be invoked by assigning the generic type with the method call: int i = 4; int j = 5; Swap(ref i, ref j); However, because the C# compiler can get the type of the parameters by calling the Swap() method, it is not necessary to assign the generic type with the method call. The generic method can be invoked as simply as non-generic methods: int i = 4; int j = 5; Swap(ref i, ref j); c05.indd 124 30-01-2014 20:11:54 Generic Methods  ❘  125 Generic Methods Example In this example, a generic method is used to accumulate all the elements of a collection. To show the features of generic methods, the following Account class, which contains Name and Balance properties, is used (code file GenericMethods/Account.cs): public class Account { public string Name { get; private set; } public decimal Balance { get; private set; } public Account(string name, Decimal balance) { this.Name = name; this.Balance = balance; } } All the accounts in which the balance should be accumulated are added to an accounts list of type List (code file GenericMethods/Program.cs): var accounts = new List() { new Account("Christian", 1500), new Account("Stephanie", 2200), new Account("Angela", 1800), new Account("Matthias", 2400) }; A traditional way to accumulate all Account objects is by looping through them with a foreach statement, as shown here. Because the foreach statement uses the IEnumerable interface to iterate the elements of a collection, the argument of the AccumulateSimple() method is of type IEnumerable. The foreach statement works with every object implementing IEnumerable. This way, the AccumulateSimple() method can be used with all collection classes that implement the interface IEnumerable. In the implementation of this method, the property Balance of the Account object is directly accessed (code file GenericMethods/Algorithm.cs): public static class Algorithm { public static decimal AccumulateSimple(IEnumerable source) { decimal sum = 0; foreach (Account a in source) { sum += a.Balance; } return sum; } } The AccumulateSimple() method is invoked like this: decimal amount = Algorithm.AccumulateSimple(accounts); Generic Methods with Constraints The problem with the first implementation is that it works only with Account objects. This can be avoided by using a generic method. c05.indd 125 30-01-2014 20:11:54 126  ❘  CHAPTER 5  Generics The second version of the Accumulate() method accepts any type that implements the interface IAccount. As you saw earlier with generic classes, generic types can be restricted with the where clause. The same clause that is used with generic classes can be used with generic methods. The parameter of the Accumulate() method is changed to IEnumerable, a generic interface that is implemented by generic collection classes (code file GenericMethods/Algorithms.cs): public static decimal Accumulate(IEnumerable source) where TAccount: IAccount { decimal sum = 0; foreach (TAccount a in source) { sum += a.Balance; } return sum; } The Account class is now refactored to implement the interface IAccount (code file GenericMethods/ Account.cs): public class Account: IAccount { //... The IAccount interface defines the read-only properties Balance and Name (code file GenericMethods/ IAccount.cs): public interface IAccount { decimal Balance { get; } string Name { get; } } The new Accumulate() method can be invoked by defining the Account type as a generic type parameter (code file GenericMethods/Program.cs): decimal amount = Algorithm.Accumulate(accounts); Because the generic type parameter can be automatically inferred by the compiler from the parameter type of the method, it is valid to invoke the Accumulate() method this way: decimal amount = Algorithm.Accumulate(accounts); Generic Methods with Delegates The requirement for the generic types to implement the interface IAccount may be too restrictive. The following example hints at how the Accumulate() method can be changed by passing a generic delegate. Chapter 8, “Delegates, Lambdas, and Events” provides all the details about how to work with generic delegates, and how to use Lambda expressions. This Accumulate() method uses two generic parameters, T1 and T2. T1 is used for the collection- implementing IEnumerable parameter, which is the first one of the methods. The second parameter uses the generic delegate Func. Here, the second and third generic parameters are of the same T2 type. A method needs to be passed that has two input parameters (T1 and T2) and a return type of T2 (code file GenericMethods/Algorithm.cs). c05.indd 126 30-01-2014 20:11:54 Generic Methods  ❘  127 public static T2 Accumulate(IEnumerable source, Func action) { T2 sum = default(T2); foreach (T1 item in source) { sum = action(item, sum); } return sum; } In calling this method, it is necessary to specify the generic parameter types because the compiler cannot infer this automatically. With the first parameter of the method, the accounts collection that is assigned is of type IEnumerable. With the second parameter, a Lambda expression is used that defines two parameters of type Account and decimal, and returns a decimal. This Lambda expression is invoked for every item by the Accumulate() method (code file GenericMethods/Program.cs): decimal amount = Algorithm.Accumulate( accounts, (item, sum) => sum += item.Balance); Don’t scratch your head over this syntax yet. The sample should give you a glimpse of the possible ways to extend the Accumulate() method. Chapter 8 covers Lambda expressions in detail. Generic Methods Specialization Generic methods can be overloaded to define specializations for specific types. This is true for methods with generic parameters as well. The Foo() method is defined in two versions. The first accepts a generic parameter; the second one is a specialized version for the int parameter. During compile time, the best match is taken. If an int is passed, then the method with the int parameter is selected. With any other parameter type, the compiler chooses the generic version of the method (code file Specialization/ Program.cs): public class MethodOverloads { public void Foo(T obj) { Console.WriteLine("Foo(T obj), obj type: {0}", obj.GetType().Name); } public void Foo(int x) { Console.WriteLine("Foo(int x)"); } public void Bar(T obj) { Foo(obj); } } The Foo() method can now be invoked with any parameter type. The sample code passes an int and a string to the method: static void Main() { var test = new MethodOverloads(); test.Foo(33); test.Foo("abc"); } c05.indd 127 30-01-2014 20:11:54 128  ❘  CHAPTER 5  Generics Running the program, you can see by the output that the method with the best match is taken: Foo(int x) Foo(T obj), obj type: String Be aware that the method invoked is defined during compile time and not runtime. This can be easily demonstrated by adding a generic Bar() method that invokes the Foo() method, passing the generic parameter value along: public class MethodOverloads { // ... public void Bar(T obj) { Foo(obj); } The Main() method is now changed to invoke the Bar() method passing an int value: static void Main() { var test = new MethodOverloads(); test.Bar(44); From the output on the console you can see that the generic Foo() method was selected by the Bar() method and not the overload with the int parameter. That’s because the compiler selects the method that is invoked by the Bar() method during compile time. Because the Bar() method defines a generic parameter, and because there’s a Foo() method that matches this type, the generic Foo() method is called. This is not changed during runtime when an int value is passed to the Bar() method: Foo(T obj), obj type: Int32 Summary This chapter introduced a very important feature of the CLR: generics. With generic classes you can create type-independent classes, and generic methods allow type-independent methods. Interfaces, structs, and delegates can be created in a generic way as well. Generics make new programming styles possible. You’ve seen how algorithms, particularly actions and predicates, can be implemented to be used with different classes — and all are type safe. Generic delegates make it possible to decouple algorithms from collections. You will see more features and uses of generics throughout this book. Chapter 8, “Delegates, Lambdas, and Events,” introduces delegates that are often implemented as generics; Chapter 10, “Collections,” provides information about generic collection classes; and Chapter 11, “Language Integrated Query,” discusses generic extension methods. The next chapter demonstrates the use of generic methods with arrays. c05.indd 128 30-01-2014 20:11:54 Arrays and Tuples WHAT’S iN THiS CHAPTER? ➤ Simple arrays ➤ Multidimensional arrays ➤ Jagged arrays ➤ The Array class ➤ Arrays as parameters ➤ Enumerations ➤ Tuples ➤ Structural comparison WRoX.Com CoDE DoWNloADS FoR THiS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤ SimpleArrays ➤ SortingSample ➤ ArraySegment ➤ YieldDemo ➤ StructuralComparison mulTiPlE oBJECTS oF THE SAmE AND DiFFERENT TyPES If you need to work with multiple objects of the same type, you can use collections (see Chapter 10, “Collections”) and arrays. C# has a special notation to declare, initialize, and use arrays. Behind the scenes, the Array class comes into play, which offers several methods to sort and fi lter the elements inside the array. Using an enumerator, you can iterate through all the elements of the array. To use multiple objects of different types, the type Tuple can be used. See the “Tuples” section later in this chapter for details about this type. 6 c06.indd 129 30-01-2014 20:12:24 130  ❘  CHAPTER 6  Arrays and Tuples Simple Arrays If you need to use multiple objects of the same type, you can use an array. An array is a data structure that contains a number of elements of the same type. Array Declaration An array is declared by defining the type of elements inside the array, followed by empty brackets and a variable name. For example, an array containing integer elements is declared like this: int[] myArray; Array Initialization After declaring an array, memory must be allocated to hold all the elements of the array. An array is a reference type, so memory on the heap must be allocated. You do this by initializing the variable of the array using the new operator, with the type and the number of elements inside the array. Here, you specify the size of the array: myArray = new int[4]; Note  Value types and reference types are covered in Chapter 3, “Objects and Types.” With this declaration and initialization, the variable myArray references four integer values that are allocated on the managed heap (see Figure 6-1). Stack myArray Managed Heap int int int int Figure 6-1 Note  An array cannot be resized after its size is specified without copying all the elements. If you don’t know how many elements should be in the array in advance, you can use a collection (see Chapter 10). Instead of using a separate line to declare and initialize an array, you can use a single line: int[] myArray = new int[4]; You can also assign values to every array element using an array initializer. Array initializers can be used only while declaring an array variable, not after the array is declared: int[] myArray = new int[4] {4, 7, 11, 2}; If you initialize the array using curly brackets, the size of the array can also be omitted, because the compiler can count the number of elements itself: c06.indd 130 30-01-2014 20:12:25 Simple Arrays  ❘  131 int[] myArray = new int[] {4, 7, 11, 2}; There’s even a shorter form using the C# compiler. Using curly brackets you can write the array declaration and initialization. The code generated from the compiler is the same as the previous result: int[] myArray = {4, 7, 11, 2}; Accessing Array Elements After an array is declared and initialized, you can access the array elements using an indexer. Arrays support only indexers that have integer parameters. With the indexer, you pass the element number to access the array. The indexer always starts with a value of 0 for the first element. Therefore, the highest number you can pass to the indexer is the number of elements minus one, because the index starts at zero. In the following example, the array myArray is declared and initialized with four integer values. The elements can be accessed with indexer values 0, 1, 2, and 3. int[] myArray = new int[] {4, 7, 11, 2}; int v1 = myArray[0]; // read first element int v2 = myArray[1]; // read second element myArray[3] = 44; // change fourth element Note  If you use a wrong indexer value where that is bigger than the length of the array, an exception of type IndexOutOfRangeException is thrown. If you don’t know the number of elements in the array, you can use the Length property, as shown in this for statement: for (int i = 0; i < myArray.Length; i++) { Console.WriteLine(myArray[i]); } Instead of using a for statement to iterate through all the elements of the array, you can also use the foreach statement: foreach (var val in myArray) { Console.WriteLine(val); } Note  The foreach statement makes use of the IEnumerable and IEnumerator interfaces, which are discussed later in this chapter. Using Reference Types In addition to being able to declare arrays of predefined types, you can also declare arrays of custom types. Let’s start with the following Person class, the properties FirstName and LastName using ­auto- implemented properties, and an override of the ToString() method from the Object class (code file SimpleArrays/Person.cs): public class Person { public string FirstName { get; set; } c06.indd 131 30-01-2014 20:12:26 132  ❘  CHAPTER 6  Arrays and Tuples public string LastName { get; set; } public override string ToString() { return String.Format("{0} {1}", FirstName, LastName); } } Declaring an array of two Person elements is similar to declaring an array of int: Person[] myPersons = new Person[2]; However, be aware that if the elements in the array are reference types, memory must be allocated for every array element. If you use an item in the array for which no memory was allocated, a NullReferenceException is thrown. Note  For information about errors and exceptions, see Chapter 16, “Errors and Exceptions.” You can allocate every element of the array by using an indexer starting from 0: myPersons[0] = new Person { FirstName="Ayrton", LastName="Senna" }; myPersons[1] = new Person { FirstName="Michael", LastName="Schumacher" }; Figure 6-2 shows the objects in the managed heap with the Person array. myPersons is a variable that is stored on the stack. This variable references an array of Person elements that is stored on the managed heap. This array has enough space for two references. Every item in the array references a Person object that is also stored in the managed heap. Similar to the int type, you can also use an array initializer with custom types: Person[] myPersons2 = { new Person { FirstName="Ayrton", LastName="Senna"}, new Person { FirstName="Michael", LastName="Schumacher"} }; Multidimensional Arrays Ordinary arrays (also known as one-dimensional arrays) are indexed by a single integer. A multidimensional array is indexed by two or more integers. Figure 6-3 shows the mathematical notation for a two-dimensional array that has three rows and three columns. The first row has the values 1, 2, and 3, and the third row has the values 7, 8, and 9. To declare this two-dimensional array with C#, you put a comma inside the brackets. The array is initialized by specifying the size of every dimension (also known as rank). Then the array elements can be accessed by using two integers with the indexer: int[,] twodim = new int[3, 3]; twodim[0, 0] = 1; myPersons Reference Person Person Reference Managed HeapStack Figure 6-2 1, 2, 3 4, 5, 6 7, 8, 9 a = Figure 6-3 c06.indd 132 30-01-2014 20:12:28 Jagged Arrays  ❘  133 twodim[0, 1] = 2; twodim[0, 2] = 3; twodim[1, 0] = 4; twodim[1, 1] = 5; twodim[1, 2] = 6; twodim[2, 0] = 7; twodim[2, 1] = 8; twodim[2, 2] = 9; Note  After declaring an array, you cannot change the rank. You can also initialize the two-dimensional array by using an array indexer if you know the values for the elements in advance. To initialize the array, one outer curly bracket is used, and every row is initialized by using curly brackets inside the outer curly brackets: int[,] twodim = { {1, 2, 3}, {4, 5, 6}, {7, 8, 9} }; Note  When using an array initializer, you must initialize every element of the array. It is not possible to defer the initialization of some values until later. By using two commas inside the brackets, you can declare a three-dimensional array: int[,,] threedim = { { { 1, 2 }, { 3, 4 } }, { { 5, 6 }, { 7, 8 } }, { { 9, 10 }, { 11, 12 } } }; Console.WriteLine(threedim[0, 1, 1]); Jagged Arrays A two-dimensional array has a rectangular size (for example, 3 × 3 elements). A jagged array provides more flexibility in sizing the array. With a jagged array every row can have a different size. Figure 6-4 contrasts a two-dimensional array that has 3 × 3 elements with a jagged array. The jagged array shown contains three rows, with the first row containing two elements, the second row containing six elements, and the third row containing three elements. Two-Dimensional Array Jagged Array 12 12 3 456 7 8 9 10 11 3 456 789 Figure 6-4 c06.indd 133 30-01-2014 20:12:30 134  ❘  CHAPTER 6  Arrays and Tuples A jagged array is declared by placing one pair of opening and closing brackets after another. To initialize the jagged array, only the size that defines the number of rows in the first pair of brackets is set. The second brackets that define the number of elements inside the row are kept empty because every row has a different number of elements. Next, the element number of the rows can be set for every row: int[][] jagged = new int[3][]; jagged[0] = new int[2] { 1, 2 }; jagged[1] = new int[6] { 3, 4, 5, 6, 7, 8 }; jagged[2] = new int[3] { 9, 10, 11 }; You can iterate through all the elements of a jagged array with nested for loops. In the outer for loop every row is iterated, and the inner for loop iterates through every element inside a row: for (int row = 0; row < jagged.Length; row++) { for (int element = 0; element < jagged[row].Length; element++) { Console.WriteLine("row: {0}, element: {1}, value: {2}", row, element, jagged[row][element]); } } The output of the iteration displays the rows and every element within the rows: row: 0, element: 0, value: 1 row: 0, element: 1, value: 2 row: 1, element: 0, value: 3 row: 1, element: 1, value: 4 row: 1, element: 2, value: 5 row: 1, element: 3, value: 6 row: 1, element: 4, value: 7 row: 1, element: 5, value: 8 row: 2, element: 0, value: 9 row: 2, element: 1, value: 10 row: 2, element: 2, value: 11 Array Class Declaring an array with brackets is a C# notation using the Array class. Using the C# syntax behind the scenes creates a new class that derives from the abstract base class Array. This makes it possible to use methods and properties that are defined with the Array class with every C# array. For example, you’ve already used the Length property or iterated through the array by using the foreach statement. By doing this, you are using the GetEnumerator() method of the Array class. Other properties implemented by the Array class are LongLength, for arrays in which the number of items doesn’t fit within an integer, and Rank, to get the number of dimensions. Let’s have a look at other members of the Array class by getting into various features. Creating Arrays The Array class is abstract, so you cannot create an array by using a constructor. However, instead of using the C# syntax to create array instances, it is also possible to create arrays by using the static CreateInstance() method. This is extremely useful if you don’t know the type of elements in advance, because the type can be passed to the CreateInstance() method as a Type object. The following example shows how to create an array of type int with a size of 5. The first argument of the CreateInstance() method requires the type of the elements, and the second argument defines the size. You can set values with the SetValue() method, and read values with the GetValue() method (code file SimpleArrays/Program.cs): c06.indd 134 30-01-2014 20:12:30 Array Class  ❘  135 Array intArray1 = Array.CreateInstance(typeof(int), 5); for (int i = 0; i < 5; i++) { intArray1.SetValue(33, i); } for (int i = 0; i < 5; i++) { Console.WriteLine(intArray1.GetValue(i)); } You can also cast the created array to an array declared as int[]: int[] intArray2 = (int[])intArray1; The CreateInstance() method has many overloads to create multidimensional arrays and to create arrays that are not 0-based. The following example creates a two-dimensional array with 2 × 3 elements. The first dimension is 1-based; the second dimension is 10-based: int[] lengths = { 2, 3 }; int[] lowerBounds = { 1, 10 }; Array racers = Array.CreateInstance(typeof(Person), lengths, lowerBounds); Setting the elements of the array, the SetValue() method accepts indices for every dimension: racers.SetValue(new Person { FirstName = "Alain", LastName = "Prost" }, index1: 1, index2: 10); racers.SetValue(new Person { FirstName = "Emerson", LastName = "Fittipaldi" }, 1, 11); racers.SetValue(new Person { FirstName = "Ayrton", LastName = "Senna" }, 1, 12); racers.SetValue(new Person { FirstName = "Michael", LastName = "Schumacher" }, 2, 10); racers.SetValue(new Person { FirstName = "Fernando", LastName = "Alonso" }, 2, 11); racers.SetValue(new Person { FirstName = "Jenson", LastName = "Button" }, 2, 12); Although the array is not 0-based, you can assign it to a variable with the normal C# notation. You just have to take care not to cross the boundaries: Person[,] racers2 = (Person[,])racers; Person first = racers2[1, 10]; Person last = racers2[2, 12]; c06.indd 135 30-01-2014 20:12:30 136  ❘  CHAPTER 6  Arrays and Tuples Copying Arrays Because arrays are reference types, assigning an array variable to another one just gives you two variables referencing the same array. For copying arrays, the array implements the interface ICloneable. The Clone() method that is defined with this interface creates a shallow copy of the array. If the elements of the array are value types, as in the following code segment, all values are copied (see Figure 6-5): int[] intArray1 = {1, 2}; int[] intArray2 = (int[])intArray1.Clone(); If the array contains reference types, only the references are copied, not the elements. Figure 6-6 shows the variables beatles and beatlesClone, where beatlesClone is created by calling the Clone() method from beatles. The Person objects that are referenced are the same for beatles and beatlesClone. If you change a property of an element of beatlesClone, you change the same object of beatles (code file SimpleArray/Program.cs): Person[] beatles = { new Person { FirstName="John", LastName="Lennon" }, new Person { FirstName="Paul", LastName="McCartney" } }; Person[] beatlesClone = (Person[])beatles.Clone(); Instead of using the Clone() method, you can use the Array.Copy() method, which also creates a shallow copy. However, there’s one important difference with Clone() and Copy(): Clone() creates a new array; with Copy() you have to pass an existing array with the same rank and enough elements. Note  If you need a deep copy of an array containing reference types, you have to iterate the array and create new objects. Sorting The Array class uses the Quicksort algorithm to sort the elements in the array. The Sort() method requires the interface IComparable to be implemented by the elements in the array. Simple types such as System.String and System.Int32 implement IComparable, so you can sort elements containing these types. With the sample program, the array name contains elements of type string, and this array can be sorted (code file SortingSample/Program.cs): string[] names = { "Christina Aguilera", "Shakira", "Beyonce", "Lady Gaga" }; Array.Sort(names); intArray1 1 2 intArray2 1 2 Figure 6-5 beatles Reference Person Reference beatlesClone Reference Person Reference Figure 6-6 c06.indd 136 30-01-2014 20:12:33 Array Class  ❘  137 foreach (var name in names) { Console.WriteLine(name); } The output of the application shows the sorted result of the array: Beyonce Christina Aguilera Lady Gaga Shakira If you are using custom classes with the array, you must implement the interface IComparable. This interface defines just one method, CompareTo(), which must return 0 if the objects to compare are equal; a value smaller than 0 if the instance should go before the object from the parameter; and a value larger than 0 if the instance should go after the object from the parameter. Change the Person class to implement the interface IComparable. The comparison is first done on the value of the LastName by using the Compare() method of the String class. If the LastName has the same value, the FirstName is compared (code file SortingSample/Person.cs): public class Person: IComparable { public int CompareTo(Person other) { if (other == null) return 1; int result = string.Compare(this.LastName, other.LastName); if (result == 0) { result = string.Compare(this.FirstName, other.FirstName); } return result; } //... Now it is possible to sort an array of Person objects by the last name (code file SortingSample/Program.cs): Person[] persons = { new Person { FirstName="Damon", LastName="Hill" }, new Person { FirstName="Niki", LastName="Lauda" }, new Person { FirstName="Ayrton", LastName="Senna" }, new Person { FirstName="Graham", LastName="Hill" } }; Array.Sort(persons); foreach (var p in persons) { Console.WriteLine(p); } Using the sort of the Person class, the output returns the names sorted by last name: Damon Hill Graham Hill Niki Lauda Ayrton Senna If the Person object should be sorted differently, or if you don’t have the option to change the class that is used as an element in the array, you can implement the interface IComparer or IComparer. These c06.indd 137 30-01-2014 20:12:33 138  ❘  CHAPTER 6  Arrays and Tuples interfaces define the method Compare(). One of these interfaces must be implemented by the class that should be compared. The IComparer interface is independent of the class to compare. That’s why the Compare() method defines two arguments that should be compared. The return value is similar to the CompareTo() method of the IComparable interface. The class PersonComparer implements the IComparer interface to sort Person objects either by firstName or by lastName. The enumeration PersonCompareType defines the different sorting options that are available with PersonComparer: FirstName and LastName. How the compare should be done is defined with the constructor of the class PersonComparer, where a PersonCompareType value is set. The Compare() method is implemented with a switch statement to compare either by LastName or by FirstName (code file SortingSample/PersonComparer.cs): public enum PersonCompareType { FirstName, LastName } public class PersonComparer: IComparer { private PersonCompareType compareType; public PersonComparer(PersonCompareType compareType) { this.compareType = compareType; } public int Compare(Person x, Person y) { if (x == null && y == null) return 0; if (x == null) return 1; if (y == null) return -1; switch (compareType) { case PersonCompareType.FirstName: return string.Compare(x.FirstName, y.FirstName); case PersonCompareType.LastName: return string.Compare(x.LastName, y.LastName); default: throw new ArgumentException("unexpected compare type"); } } } Now you can pass a PersonComparer object to the second argument of the Array.Sort() method. Here, the persons are sorted by first name (code file SortingSample/Program.cs): Array.Sort(persons, new PersonComparer(PersonCompareType.FirstName)); foreach (var p in persons) { Console.WriteLine(p); } The persons array is now sorted by first name: Ayrton Senna Damon Hill Graham Hill Niki Lauda c06.indd 138 30-01-2014 20:12:33 Arrays as Parameters  ❘  139 Note  The Array class also offers Sort methods that require a delegate as an argument. With this argument you can pass a method to do the comparison of two objects, rather than rely on the IComparable or IComparer interfaces. Chapter 8, “Delegates, Lambdas, and Events,” discusses how to use delegates. Arrays as Parameters Arrays can be passed as parameters to methods, and returned from methods. Returning an array, you just have to declare the array as the return type, as shown with the following method GetPersons(): static Person[] GetPersons() { return new Person[] { new Person { FirstName="Damon", LastName="Hill" }, new Person { FirstName="Niki", LastName="Lauda" }, new Person { FirstName="Ayrton", LastName="Senna" }, new Person { FirstName="Graham", LastName="Hill" } }; } Passing arrays to a method, the array is declared with the parameter, as shown with the method DisplayPersons(): static void DisplayPersons(Person[] persons) { //... Array Covariance With arrays, covariance is supported. This means that an array can be declared as a base type and elements of derived types can be assigned to the elements. For example, you can declare a parameter of type object[] as shown and pass a Person[] to it: static void DisplayArray(object[] data) { //... } Note  Array covariance is only possible with reference types, not with value types. In addition, array covariance has an issue that can only be resolved with runtime exceptions. If you assign a Person array to an object array, the object array can then be used with anything that derives from the object. The compiler accepts, for example, passing a string to array elements. However, because a Person array is referenced by the object array, a runtime exception, ArrayTypeMismatchException, occurs. ArraySegment The struct ArraySegment represents a segment of an array. If you are working with a large array, and different methods work on parts of the array, you could copy the array part to the different methods. Instead of creating multiple arrays, it is more efficient to use one array and pass the complete array to c06.indd 139 30-01-2014 20:12:33 140  ❘  CHAPTER 6  Arrays and Tuples the methods. The methods should only use a part of the array. For this, you can pass the offset into the array and the count of elements that the method should use in addition to the array. This way, at least three parameters are needed. When using an array segment, just a single parameter is needed. The ArraySegment structure contains information about the segment (the offset and count). The method SumOfSegments takes an array of ArraySegment elements to calculate the sum of all the integers that are defined with the segments and returns the sum (code file ArraySegmentSample/ Program.cs): static int SumOfSegments(ArraySegment[] segments) { int sum = 0; foreach (var segment in segments) { for (int i = segment.Offset; i < segment.Offset + segment.Count; i++) { sum += segment.Array[i]; } } return sum; } This method is used by passing an array of segments. The first array element references three elements of ar1 starting with the first element; the second array element references three elements of ar2 starting with the fourth element: int[] ar1 = { 1, 4, 5, 11, 13, 18 }; int[] ar2 = { 3, 4, 5, 18, 21, 27, 33 }; var segments = new ArraySegment[2] { new ArraySegment(ar1, 0, 3), new ArraySegment(ar2, 3, 3) }; var sum = SumOfSegments(segments); Note  Array segments don’t copy the elements of the originating array. Instead, the originating array can be accessed through ArraySegment. If elements of the array segment are changed, the changes can be seen in the original array. Enumerations By using the foreach statement you can iterate elements of a collection (see Chapter 10, “Collections”) without needing to know the number of elements inside the collection. The foreach statement uses an enumerator. Figure 6-7 shows the relationship between the client invoking the foreach method and the collection. The array or collection implements the IEnumerable interface with the GetEnumerator() method. The GetEnumerator() method returns an enumerator implementing the IEnumerator interface. The interface IEnumerator is then used by the foreach statement to iterate through the collection. Client Enumerator IEnumerator IEnumerable Collection Figure 6-7 c06.indd 140 30-01-2014 20:12:35 Enumerations  ❘  141 Note  The GetEnumerator() method is defined with the interface IEnumerable. The foreach statement doesn’t really need this interface implemented in the collection class. It’s enough to have a method with the name GetEnumerator() that returns an object implementing the IEnumerator interface. IEnumerator Interface The foreach statement uses the methods and properties of the IEnumerator interface to iterate all elements in a collection. For this, IEnumerator defines the property Current to return the element where the cursor is positioned, and the method MoveNext() to move to the next element of the collection. MoveNext() returns true if there’s an element, and false if no more elements are available. The generic version of this interface IEnumerator derives from the interface IDisposable and thus defines a Dispose() method to clean up resources allocated by the enumerator. Note  The IEnumerator interface also defines the Reset() method for COM interoperability. Many .NET enumerators implement this by throwing an exception of type NotSupportedException. foreach Statement The C# foreach statement is not resolved to a foreach statement in the IL code. Instead, the C# compiler converts the foreach statement to methods and properties of the IEnumerator interface. Here’s a simple foreach statement to iterate all elements in the persons array and display them person by person: foreach (var p in persons) { Console.WriteLine(p); } The foreach statement is resolved to the following code segment. First, the GetEnumerator() method is invoked to get an enumerator for the array. Inside a while loop, as long as MoveNext() returns true, the elements of the array are accessed using the Current property: IEnumerator enumerator = persons.GetEnumerator(); while (enumerator.MoveNext()) { Person p = enumerator.Current; Console.WriteLine(p); } yield Statement Since the first release of C#, it has been easy to iterate through collections by using the foreach statement. With C# 1.0, it was still a lot of work to create an enumerator. C# 2.0 added the yield statement for creating enumerators easily. The yield return statement returns one element of a collection and moves the position to the next element, and yield break stops the iteration. The next example shows the implementation of a simple collection using the yield return statement. The class HelloCollection contains the method GetEnumerator(). The implementation of the c06.indd 141 30-01-2014 20:12:35 142  ❘  CHAPTER 6  Arrays and Tuples GetEnumerator() method contains two yield return statements where the strings Hello and World are returned (code file YieldDemo/Program.cs): using System; using System.Collections; namespace Wrox.ProCSharp.Arrays { public class HelloCollection { public IEnumerator GetEnumerator() { yield return "Hello"; yield return "World"; } } Note  A method or property that contains yield statements is also known as an iterator block. An iterator block must be declared to return an IEnumerator or IEnumerable interface, or the generic versions of these interfaces. This block may contain multiple yield return or yield break statements; a return statement is not allowed. Now it is possible to iterate through the collection using a foreach statement: public void HelloWorld() { var helloCollection = new HelloCollection(); foreach (var s in helloCollection) { Console.WriteLine(s); } } } With an iterator block, the compiler generates a yield type, including a state machine, as shown in the following code segment. The yield type implements the properties and methods of the interfaces IEnumerator and IDisposable. In the example, you can see the yield type as the inner class Enumerator. The GetEnumerator() method of the outer class instantiates and returns a new yield type. Within the yield type, the variable state defines the current position of the iteration and is changed every time the method MoveNext() is invoked. MoveNext() encapsulates the code of the iterator block and sets the value of the current variable so that the Current property returns an object depending on the position: public class HelloCollection { public IEnumerator GetEnumerator() { return new Enumerator(0); } public class Enumerator: IEnumerator, IEnumerator, IDisposable { private int state; private string current; public Enumerator(int state) { c06.indd 142 30-01-2014 20:12:35 Enumerations  ❘  143 this.state = state; } bool System.Collections.IEnumerator.MoveNext() { switch (state) { case 0: current = "Hello"; state = 1; return true; case 1: current = "World"; state = 2; return true; case 2: break; } return false; } void System.Collections.IEnumerator.Reset() { throw new NotSupportedException(); } string System.Collections.Generic.IEnumerator.Current { get { return current; } } object System.Collections.IEnumerator.Current { get { return current; } } void IDisposable.Dispose() { } } } Note  Remember that the yield statement produces an enumerator, and not just a list filled with items. This enumerator is invoked by the foreach statement. As each item is accessed from the foreach, the enumerator is accessed. This makes it possible to iterate through huge amounts of data without reading all the data into memory in one turn. Different Ways to Iterate Through Collections In a slightly larger and more realistic way than the Hello World example, you can use the yield return statement to iterate through a collection in different ways. The class MusicTitles enables iterating the titles c06.indd 143 30-01-2014 20:12:35 144  ❘  CHAPTER 6  Arrays and Tuples in a default way with the GetEnumerator() method, in reverse order with the Reverse() method, and through a subset with the Subset() method (code file YieldDemo/MusicTitles.cs): public class MusicTitles { string[] names = { "Tubular Bells", "Hergest Ridge", "Ommadawn", "Platinum" }; public IEnumerator GetEnumerator() { for (int i = 0; i < 4; i++) { yield return names[i]; } } public IEnumerable Reverse() { for (int i = 3; i >= 0; i--) { yield return names[i]; } } public IEnumerable Subset(int index, int length) { for (int i = index; i < index + length; i++) { yield return names[i]; } } } Note  The default iteration supported by a class is the GetEnumerator() method, which is defined to return IEnumerator. Named iterations return IEnumerable. The client code to iterate through the string array first uses the GetEnumerator() method, which you don’t have to write in your code because it is used by default with the implementation of the foreach statement. Then the titles are iterated in reverse, and finally a subset is iterated by passing the index and number of items to iterate to the Subset() method (code file YieldDemo/Program.cs): var titles = new MusicTitles(); foreach (var title in titles) { Console.WriteLine(title); } Console.WriteLine(); Console.WriteLine("reverse"); foreach (var title in titles.Reverse()) { Console.WriteLine(title); } Console.WriteLine(); Console.WriteLine("subset"); c06.indd 144 30-01-2014 20:12:36 Enumerations  ❘  145 foreach (var title in titles.Subset(2, 2)) { Console.WriteLine(title); } Returning Enumerators with Yield Return With the yield statement you can also do more complex things, such as return an enumerator from yield return. Using the following Tic-Tac-Toe game as an example, players alternate putting a cross or a circle in one of nine fields. These moves are simulated by the GameMoves class. The methods Cross() and Circle() are the iterator blocks for creating iterator types. The variables cross and circle are set to Cross() and Circle() inside the constructor of the GameMoves class. By setting these fields the methods are not invoked, but they are set to the iterator types that are defined with the iterator blocks. Within the Cross() iterator block, information about the move is written to the console and the move number is incremented. If the move number is higher than 8, the iteration ends with yield break; otherwise, the enumerator object of the circle yield type is returned with each iteration. The Circle() iterator block is very similar to the Cross() iterator block; it just returns the cross iterator type with each iteration (code file YieldDemo/ GameMoves.cs): public class GameMoves { private IEnumerator cross; private IEnumerator circle; public GameMoves() { cross = Cross(); circle = Circle(); } private int move = 0; const int MaxMoves = 9; public IEnumerator Cross() { while (true) { Console.WriteLine("Cross, move {0}", move); if (++move >= MaxMoves) yield break; yield return circle; } } public IEnumerator Circle() { while (true) { Console.WriteLine("Circle, move {0}", move); if (++move >= MaxMoves) yield break; yield return cross; } } } From the client program, you can use the class GameMoves as follows. The first move is set by setting enumerator to the enumerator type returned by game.Cross(). In a while loop, enumerator.MoveNext() is called. The first time this is invoked, the Cross() method is called, which returns the other enumerator c06.indd 145 30-01-2014 20:12:36 146  ❘  CHAPTER 6  Arrays and Tuples with a yield statement. The returned value can be accessed with the Current property and is set to the enumerator variable for the next loop: var game = new GameMoves(); IEnumerator enumerator = game.Cross(); while (enumerator.MoveNext()) { enumerator = enumerator.Current as IEnumerator; } The output of this program shows alternating moves until the last move: Cross, move 0 Circle, move 1 Cross, move 2 Circle, move 3 Cross, move 4 Circle, move 5 Cross, move 6 Circle, move 7 Cross, move 8 Tuples Whereas arrays combine objects of the same type, tuples can combine objects of different types. Tuples have their origin in functional programming languages such as F# where they are used often. With the .NET Framework, tuples are available for all .NET languages. The .NET Framework defines eight generic Tuple classes (since version 4.0) and one static Tuple class that act as a factory of tuples. The different generic Tuple classes support a different number of elements — e.g., Tuple contains one element, Tuple contains two elements, and so on. The method Divide() demonstrates returning a tuple with two members: Tuple. The parameters of the generic class define the types of the members, which are both integers. The tuple is created with the static Create() method of the static Tuple class. Again, the generic parameters of the Create() method define the type of tuple that is instantiated. The newly created tuple is initialized with the result and reminder variables to return the result of the division (code file TupleSamle/Program.cs): public static Tuple Divide(int dividend, int divisor) { int result = dividend / divisor; int reminder = dividend % divisor; return Tuple.Create(result, reminder); } The following example demonstrates invoking the Divide() method. The items of the tuple can be accessed with the properties Item1 and Item2: var result = Divide(5, 2); Console.WriteLine("result of division: {0}, reminder: {1}", result.Item1, result.Item2); If you have more than eight items that should be included in a tuple, you can use the Tuple class definition with eight parameters. The last template parameter is named TRest to indicate that you must pass a tuple itself. That way you can create tuples with any number of parameters. c06.indd 146 30-01-2014 20:12:36 Structural Comparison  ❘  147 The following example demonstrates this functionality: public class Tuple Here, the last template parameter is a tuple type itself, so you can create a tuple with any number of items: var tuple = Tuple.Create>("Stephanie", "Alina", "Nagel", 2009, 6, 2, 1.37, Tuple.Create(52, 3490)); Structural Comparison Both arrays and tuples implement the interfaces IStructuralEquatable and IStructuralComparable. These interfaces are new since .NET 4 and compare not only references but also the content. This interface is implemented explicitly, so it is necessary to cast the arrays and tuples to this interface on use. IStructuralEquatable is used to compare whether two tuples or arrays have the same content; IStructuralComparable is used to sort tuples or arrays. With the sample demonstrating IStructuralEquatable, the Person class implementing the interface IEquatable is used. IEquatable defines a strongly typed Equals() method where the values of the FirstName and LastName properties are compared (code file StructuralComparison/Person.cs): public class Person: IEquatable { public int Id { get; private set; } public string FirstName { get; set; } public string LastName { get; set; } public override string ToString() { return String.Format("{0}, {1} {2}", Id, FirstName, LastName); } public override bool Equals(object obj) { if (obj == null) return base.Equals(obj); return Equals(obj as Person); } public override int GetHashCode() { return Id.GetHashCode(); } public bool Equals(Person other) { if (other == null) return base.Equals(other); return this.Id == other.Id && this.FirstName == other.FirstName && this.LastName == other.LastName; } } Now two arrays containing Person items are created. Both arrays contain the same Person object with the variable name janet, and two different Person objects that have the same content. The comparison operator != returns true because there are indeed two different arrays referenced from two variable names, c06.indd 147 30-01-2014 20:12:36 148  ❘  CHAPTER 6  Arrays and Tuples persons1 and persons2. Because the Equals() method with one parameter is not overridden by the Array class, the same happens as with the == operator to compare the references, and they are not the same (code file StructuralComparison/Program.cs): var janet = new Person { FirstName = "Janet", LastName = "Jackson" }; Person[] persons1 = { new Person { FirstName = "Michael", LastName = "Jackson" }, janet }; Person[] persons2 = { new Person { FirstName = "Michael", LastName = "Jackson" }, janet }; if (persons1 != persons2) Console.WriteLine("not the same reference"); Invoking the Equals() method defined by the IStructuralEquatable interface — that is, the method with the first parameter of type object and the second parameter of type IEqualityComparer — you can define how the comparison should be done by passing an object that implements IEqualityComparer. A default implementation of the IEqualityComparer is done by the EqualityComparer class. This implementation checks whether the type implements the interface IEquatable, and invokes the IEquatable.Equals() method. If the type does not implement IEquatable, the Equals() method from the base class Object is invoked to do the comparison. Person implements IEquatable, where the content of the objects is compared, and the arrays indeed contain the same content: if ((persons1 as IStructuralEquatable).Equals(persons2, EqualityComparer.Default)) { Console.WriteLine("the same content"); } Next, you’ll see how the same thing can be done with tuples. Here, two tuple instances are created that have the same content. Of course, because the references t1 and t2 reference two different objects, the comparison operator != returns true: var t1 = Tuple.Create(1, "Stephanie"); var t2 = Tuple.Create(1, "Stephanie"); if (t1 != t2) Console.WriteLine("not the same reference to the tuple"); The Tuple<> class offers two Equals() methods: one that is overridden from the Object base class with an object as parameter, and the second that is defined by the IStructuralEqualityComparer interface with object and IEqualityComparer as parameters. Another tuple can be passed to the first method as shown. This method uses EqualityComparer.Default to get an ObjectEqualityComparer for the comparison. This way, every item of the tuple is compared by invoking the Object.Equals() method. If every item returns true, the result of the Equals() method is true, which is the case here with the same int and string values: c06.indd 148 30-01-2014 20:12:36 Summary  ❘  149 if (t1.Equals(t2)) Console.WriteLine("the same content"); You can also create a custom IEqualityComparer, as shown in the following example, with the class TupleComparer. This class implements the two methods Equals() and GetHashCode() of the IEqualityComparer interface: class TupleComparer: IEqualityComparer { public new bool Equals(object x, object y) { return x.Equals(y); } public int GetHashCode(object obj) { return obj.GetHashCode(); } } Note  Implementation of the Equals() method of the IEqualityComparer interface requires the new modifier or an implicit interface implementation because the base class Object defines a static Equals() method with two parameters as well. The TupleComparer is used, passing a new instance to the Equals() method of the Tuple class. The Equals() method of the Tuple class invokes the Equals() method of the TupleComparer for every item to be compared. Therefore, with the Tuple class, the TupleComparer is invoked two times to check whether all items are equal: if (t1.Equals(t2, new TupleComparer())) Console.WriteLine("equals using TupleComparer"); Summary In this chapter, you’ve seen the C# notation to create and use simple, multidimensional, and jagged arrays. The Array class is used behind the scenes of C# arrays, enabling you to invoke properties and methods of this class with array variables. You’ve seen how to sort elements in the array by using the IComparable and IComparer interfaces; and you’ve learned how to create and use enumerators, the interfaces IEnumerable and IEnumerator, and the yield statement. Finally, you have seen how to unite objects of the same type to an array, and objects of different types to a tuple. The next chapter focuses on operators and casts. c06.indd 149 30-01-2014 20:12:36 c06.indd 150 30-01-2014 20:12:36 Operators and Casts WHAT’S in THiS CHAPTER? ➤➤ Operators in C# ➤➤ The idea of equality when dealing with reference and value types ➤➤ Data conversion between primitive data types ➤➤ Converting value types to reference types using boxing ➤➤ Converting between reference types by casting ➤➤ Overloading the standard operators for custom types ➤➤ Adding cast operators to custom types WRoX.Com CodE doWnloAdS foR THiS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ SimpleCurrency ➤➤ SimpleCurrency2 ➤➤ VectorStruct ➤➤ VectorStructMoreOverloads oPERAToRS And CASTS The preceding chapters have covered most of what you need to start writing useful programs using C#. This chapter completes the discussion of the essential language elements and illustrates some powerful aspects of C# that enable you to extend its capabilities. oPERAToRS Although most of C#’s operators should be familiar to C and C++ developers, this section discusses the most important operators for the benefi t of new programmers and Visual Basic converts, and sheds light on some of the changes introduced with C#. 7 c07.indd 151 30-01-2014 20:13:13 152  ❘  CHAPTER 7  Operators and Casts C# supports the operators listed in the following table: Category Operator Arithmetic + – * / % Logical & | ^ ~ && || ! String concatenation + Increment and decrement ++ –– Bit shifting << >> Comparison == != < > <= >= Assignment = += -= *= /= %= &= |= ^= <<= >>= Member access (for objects and structs) . Indexing (for arrays and indexers) [] Cast () Conditional (the ternary operator) ?: Delegate concatenation and removal (discussed in Chapter 8, “Delegates, Lambdas, and Events”) + - Object creation new Type information sizeof is typeof as Overflow exception control checked unchecked Indirection and address [] Namespace alias qualifier (discussed in Chapter 2, “Core C#”) :: Null coalescing operator ?? However, note that four specific operators (sizeof, *, ->, and &, listed in the following table) are available only in unsafe code (code that bypasses C#’s type-safety checking), which is discussed in Chapter 14, “Memory Management and Pointers.” It is also important to note that the sizeof operator keywords, when used with the very early versions of the .NET Framework 1.0 and 1.1, required the unsafe mode. This is not a requirement since the .NET Framework 2.0. Category Operator Operator keywords sizeof (for .NET Framework versions 1.0 and 1.1 only) Operators * -> & One of the biggest pitfalls to watch out for when using C# operators is that, as with other C-style languages, C# uses different operators for assignment (=) and comparison (==). For instance, the following statement means “let x equal three”: x = 3; If you now want to compare x to a value, you need to use the double equals sign ==: if (x == 3) { } Fortunately, C#’s strict type-safety rules prevent the very common C error whereby assignment is performed instead of comparison in logical statements. This means that in C# the following statement will generate a compiler error: c07.indd 152 30-01-2014 20:13:13 Operators  ❘  153 if (x = 3) { } Visual Basic programmers who are accustomed to using the ampersand (&) character to concatenate strings will have to make an adjustment. In C#, the plus sign (+) is used instead for concatenation, whereas the & symbol denotes a bitwise AND between two different integer values. The pipe symbol, |, enables you to perform a bitwise OR between two integers. Visual Basic programmers also might not recognize the modulus (%) arithmetic operator. This returns the remainder after division, so, for example, x % 5 returns 2 if x is equal to 7. You will use few pointers in C#, and therefore few indirection operators. More specifically, the only place you will use them is within blocks of unsafe code, because that is the only place in C# where pointers are allowed. Pointers and unsafe code are discussed in Chapter 14. Operator Shortcuts The following table shows the full list of shortcut assignment operators available in C#: Shortcut Operator Equivalent To x++, ++x x = x + 1 x--, --x x = x – 1 x += y x = x + y x -= y x = x - y x *= y x = x * y x /= y x = x / y x %= y x = x % y x >>= y x = x >> y x <<= y x = x << y x &= y x = x & y x |= y x = x | y You may be wondering why there are two examples each for the ++ increment and the -- decrement operators. Placing the operator before the expression is known as a prefix; placing the operator after the expression is known as a postfix. Note that there is a difference in the way they behave. The increment and decrement operators can act both as entire expressions and within expressions. When used by themselves, the effect of both the prefix and postfix versions is identical and corresponds to the statement x = x + 1. When used within larger expressions, the prefix operator will increment the value of x before the expression is evaluated; in other words, x is incremented and the new value is used in the expression. Conversely, the postfix operator increments the value of x after the expression is evaluated — the expression is evaluated using the original value of x. The following example uses the increment operator (++) as an example to demonstrate the difference between the prefix and postfix behavior: int x = 5; if (++x == 6) // true – x is incremented to 6 before the evaluation { Console.WriteLine("This will execute"); } if (x++ == 7) // false – x is incremented to 7 after the evaluation { Console.WriteLine("This won't"); } c07.indd 153 30-01-2014 20:13:13 154  ❘  CHAPTER 7  Operators and Casts The first if condition evaluates to true because x is incremented from 5 to 6 before the expression is evaluated. The condition in the second if statement is false, however, because x is incremented to 7 only after the entire expression has been evaluated (while x == 6). The prefix and postfix operators --x and x-- behave in the same way, but decrement rather than increment the operand. The other shortcut operators, such as += and -=, require two operands, and are used to modify the value of the first operand by performing an arithmetic, logical, or bitwise operation on it. For example, the next two lines are equivalent: x += 5; x = x + 5; The following sections look at some of the primary and cast operators that you will frequently use within your C# code. The Conditional Operator (==) The conditional operator (?:), also known as the ternary operator, is a shorthand form of the if...else construction. It gets its name from the fact that it involves three operands. It allows you to evaluate a condition, returning one value if that condition is true, or another value if it is false. The syntax is as follows: condition ? true_value: false_value Here, condition is the Boolean expression to be evaluated, true_value is the value that will be returned if condition is true, and false_value is the value that will be returned otherwise. When used sparingly, the conditional operator can add a dash of terseness to your programs. It is especially handy for providing one of a couple of arguments to a function that is being invoked. You can use it to quickly convert a Boolean value to a string value of true or false. It is also handy for displaying the correct singular or plural form of a word: int x = 1; string s = x + " "; s += (x == 1 ? "man": "men"); Console.WriteLine(s); This code displays 1 man if x is equal to one but will display the correct plural form for any other number. Note, however, that if your output needs to be localized to different languages, you have to write more sophisticated routines to take into account the different grammatical rules of different languages. The checked and unchecked Operators Consider the following code: byte b = 255; b++; Console.WriteLine(b.ToString()); The byte data type can hold values only in the range 0 to 255, so incrementing the value of b causes an overflow. How the CLR handles this depends on a number of issues, including compiler options; so whenever there’s a risk of an unintentional overflow, you need some way to ensure that you get the result you want. To do this, C# provides the checked and unchecked operators. If you mark a block of code as checked, the CLR will enforce overflow checking, throwing an OverflowException if an overflow occurs. The following changes the preceding code to include the checked operator: byte b = 255; checked { b++; } Console.WriteLine(b.ToString()); c07.indd 154 30-01-2014 20:13:14 Operators  ❘  155 When you try to run this code, you will get an error message like this: Unhandled Exception: System.OverflowException: Arithmetic operation resulted in an overflow at Wrox.ProCSharp.Basics.OverflowTest.Main(String[] args) Note  You can enforce overflow checking for all unmarked code in your program by -specifying the /checked compiler option. If you want to suppress overflow checking, you can mark the code as unchecked: byte b = 255; unchecked { b++; } Console.WriteLine(b.ToString()); In this case, no exception will be raised but you will lose data because the byte type cannot hold a value of 256, the overflowing bits will be discarded, and your b variable will hold a value of zero (0). Note that unchecked is the default behavior. The only time you are likely to need to explicitly use the unchecked keyword is when you need a few unchecked lines of code inside a larger block that you have explicitly marked as checked. The is Operator The is operator allows you to check whether an object is compatible with a specific type. The phrase “is compatible” means that an object either is of that type or is derived from that type. For example, to check whether a variable is compatible with the object type, you could use the following bit of code: int i = 10; if (i is object) { Console.WriteLine("i is an object"); } int, like all C# data types, inherits from object; therefore, the expression i is object evaluates to true in this case, and the appropriate message will be displayed. The as Operator The as operator is used to perform explicit type conversions of reference types. If the type being converted is compatible with the specified type, conversion is performed successfully. However, if the types are incompatible, the as operator returns the value null. As shown in the following code, attempting to convert an object reference to a string will return null if the object reference does not actually refer to a string instance: object o1 = "Some String"; object o2 = 5; string s1 = o1 as string; // s1 = "Some String" string s2 = o2 as string; // s2 = null The as operator allows you to perform a safe type conversion in a single step without the need to first test the type using the is operator and then perform the conversion. The sizeof Operator You can determine the size (in bytes) required on the stack by a value type using the sizeof operator: Console.WriteLine(sizeof(int)); c07.indd 155 30-01-2014 20:13:14 156  ❘  CHAPTER 7  Operators and Casts This will display the number 4, because an int is 4 bytes long. If you are using the sizeof operator with complex types (and not primitive types), you need to block the code within an unsafe block as illustrated here: unsafe { Console.WriteLine(sizeof(Customer)); } Chapter 14 looks at unsafe code in more detail. The typeof Operator The typeof operator returns a System.Type object representing a specified type. For example, typeof(string) will return a Type object representing the System.String type. This is useful when you want to use reflection to find information about an object dynamically. For more information, see Chapter 15, “Reflection.” Nullable Types and Operators Looking at the Boolean type, you have a true or false value that you can assign to this type. However, what if you wanted to define the value of the type as undefined? This is where using nullable types can add a distinct value to your applications. If you use nullable types in your programs, you must always consider the effect a null value can have when used in conjunction with the various operators. Usually, when using a unary or binary operator with nullable types, the result will be null if one or both of the operands is null. For example: int? a = null; int? b = a + 4; // b = null int? c = a * 5; // c = null However, when comparing nullable types, if only one of the operands is null, the comparison will always equate to false. This means that you cannot assume a condition is true just because its opposite is false, as often happens in programs using non-nullable types. For example: int? a = null; int? b = -5; if (a > = b) Console.WriteLine("a > = b"); else Console.WriteLine("a < b"); Note  The possibility of a null value means that you cannot freely combine nullable and non-nullable types in an expression. This is discussed in the section “Type Conversions” later in this chapter. The Null Coalescing Operator The null coalescing operator (??) provides a shorthand mechanism to cater to the possibility of null values when working with nullable and reference types. The operator is placed between two operands — the first operand must be a nullable type or reference type, and the second operand must be of the same type as the first or of a type that is implicitly convertible to the type of the first operand. The null coalescing operator evaluates as follows: c07.indd 156 30-01-2014 20:13:14 Type Safety  ❘  157 ➤➤ If the first operand is not null, then the overall expression has the value of the first operand. ➤➤ If the first operand is null, then the overall expression has the value of the second operand. For example: int? a = null; int b; b = a ?? 10; // b has the value 10 a = 3; b = a ?? 10; // b has the value 3 If the second operand cannot be implicitly converted to the type of the first operand, a compile-time error is generated. Operator Precedence The following table shows the order of precedence of the C# operators. The operators at the top of the table are those with the highest precedence (that is, the ones evaluated first in an expression containing multiple operators). Group Operators Primary () . [] x++ x-- new typeof sizeof checked unchecked Unary +  —  ! ~ ++x --x and casts Multiplication/division * / % Addition/subtraction + - Bitwise shift operators << >> Relational < ><= >= is as Comparison == != Bitwise AND & Bitwise XOR ^ Bitwise OR | Boolean AND && Boolean OR || Conditional operator ?: Assignment = += -= *= /= %= &= |= ^= <<= >>= >>>= Note  In complex expressions, avoid relying on operator precedence to produce the correct result. Using parentheses to specify the order in which you want operators applied clarifies your code and prevents potential confusion. Type Safety Chapter 1, “.NET Architecture,” noted that the Intermediate Language (IL) enforces strong type safety upon its code. Strong typing enables many of the services provided by .NET, including security and language interoperability. As you would expect from a language compiled into IL, C# is also strongly typed. Among other things, this means that data types are not always seamlessly interchangeable. This section looks at conversions between primitive types. c07.indd 157 30-01-2014 20:13:14 158  ❘  CHAPTER 7  Operators and Casts Note  C# also supports conversions between different reference types and allows you to define how data types that you create behave when converted to and from other types. Both of these topics are discussed later in this chapter. Generics, however, enable you to avoid some of the most common situations in which you would need to perform type conversions. See Chapter 5, “Generics” and Chapter 10, “Collections,” for details. Type Conversions Often, you need to convert data from one type to another. Consider the following code: byte value1 = 10; byte value2 = 23; byte total; total = value1 + value2; Console.WriteLine(total); When you attempt to compile these lines, you get the following error message: Cannot implicitly convert type 'int' to 'byte' The problem here is that when you add 2 bytes together, the result will be returned as an int, not another byte. This is because a byte can contain only 8 bits of data, so adding 2 bytes together could very easily result in a value that cannot be stored in a single byte. If you want to store this result in a byte variable, you have to convert it back to a byte. The following sections discuss two conversion mechanisms supported by C# — implicit and explicit. Implicit Conversions Conversion between types can normally be achieved automatically (implicitly) only if you can guarantee that the value is not changed in any way. This is why the previous code failed; by attempting a conversion from an int to a byte, you were potentially losing 3 bytes of data. The compiler won’t let you do that unless you explicitly specify that’s what you want to do. If you store the result in a long instead of a byte, however, you will have no problems: byte value1 = 10; byte value2 = 23; long total; // this will compile fine total = value1 + value2; Console.WriteLine(total); Your program has compiled with no errors at this point because a long holds more bytes of data than a byte, so there is no risk of data being lost. In these circumstances, the compiler is happy to make the conversion for you, without your needing to ask for it explicitly. The following table shows the implicit type conversions supported in C#: From To sbyte short, int, long, float, double, decimal, BigInteger byte short, ushort, int, uint, long, ulong, float, double, decimal, BigInteger short int, long, float, double, decimal, BigInteger ushort int, uint, long, ulong, float, double, decimal, BigInteger int long, float, double, decimal, BigInteger uint long, ulong, float, double, decimal, BigInteger c07.indd 158 30-01-2014 20:13:14 Type Safety  ❘  159 From To long, ulong float, double, decimal, BigInteger float double, BigInteger char ushort, int, uint, long, ulong, float, double, decimal, BigInteger As you would expect, you can perform implicit conversions only from a smaller integer type to a larger one, not from larger to smaller. You can also convert between integers and floating-point values; however, the rules are slightly different here. Though you can convert between types of the same size, such as int/uint to float and long/ulong to double, you can also convert from long/ulong back to float. You might lose 4 bytes of data doing this, but it only means that the value of the float you receive will be less precise than if you had used a double; the compiler regards this as an acceptable possible error because the magnitude of the value is not affected. You can also assign an unsigned variable to a signed variable as long as the value limits of the unsigned type fit between the limits of the signed variable. Nullable types introduce additional considerations when implicitly converting value types: ➤➤ Nullable types implicitly convert to other nullable types following the conversion rules described for non-nullable types in the previous table; that is, int? implicitly converts to long?, float?, double?, and decimal?. ➤➤ Non-nullable types implicitly convert to nullable types according to the conversion rules described in the preceding table; that is, int implicitly converts to long?, float?, double?, and decimal?. ➤➤ Nullable types do not implicitly convert to non-nullable types; you must perform an explicit conversion as described in the next section. That’s because there is a chance that a nullable type will have the value null, which cannot be represented by a non-nullable type. Explicit Conversions Many conversions cannot be implicitly made between types, and the compiler will return an error if any are attempted. These are some of the conversions that cannot be made implicitly: ➤➤ int to short — Data loss is possible. ➤➤ int to uint — Data loss is possible. ➤➤ uint to int — Data loss is possible. ➤➤ float to int — Everything is lost after the decimal point. ➤➤ Any numeric type to char — Data loss is possible. ➤➤ decimal to any numeric type — The decimal type is internally structured differently from both integers and floating-point numbers. ➤➤ int? to int — The nullable type may have the value null. However, you can explicitly carry out such conversions using casts. When you cast one type to another, you deliberately force the compiler to make the conversion. A cast looks like this: long val = 30000; int i = (int)val; // A valid cast. The maximum int is 2147483647 You indicate the type to which you are casting by placing its name in parentheses before the value to be converted. If you are familiar with C, this is the typical syntax for casts. If you are familiar with the C++ special cast keywords such as static_cast, note that these do not exist in C#; you have to use the older C-type syntax. Casting can be a dangerous operation to undertake. Even a simple cast from a long to an int can cause problems if the value of the original long is greater than the maximum value of an int: long val = 3000000000; int i = (int)val; // An invalid cast. The maximum int is 2147483647 c07.indd 159 30-01-2014 20:13:15 160  ❘  CHAPTER 7  Operators and Casts In this case, you will not get an error, but nor will you get the result you expect. If you run this code and output the value stored in i, this is what you get: -1294967296 It is good practice to assume that an explicit cast will not return the results you expect. As shown earlier, C# provides a checked operator that you can use to test whether an operation causes an arithmetic overflow. You can use the checked operator to confirm that a cast is safe and to force the runtime to throw an overflow exception if it is not: long val = 3000000000; int i = checked((int)val); Bearing in mind that all explicit casts are potentially unsafe, take care to include code in your application to deal with possible failures of the casts. Chapter 16, “Errors and Exceptions,” introduces structured exception handling using the try and catch statements. Using casts, you can convert most primitive data types from one type to another; for example, in the following code, the value 0.5 is added to price, and the total is cast to an int: double price = 25.30; int approximatePrice = (int)(price + 0.5); This gives the price rounded to the nearest dollar. However, in this conversion, data is lost — namely, everything after the decimal point. Therefore, such a conversion should never be used if you want to continue to do more calculations using this modified price value. However, it is useful if you want to output the approximate value of a completed or partially completed calculation — if you don’t want to bother the user with a lot of figures after the decimal point. This example shows what happens if you convert an unsigned integer into a char: ushort c = 43; char symbol = (char)c; Console.WriteLine(symbol); The output is the character that has an ASCII number of 43, the + sign. You can try any kind of conversion you want between the numeric types (including char) and it will work, such as converting a decimal into a char, or vice versa. Converting between value types is not restricted to isolated variables, as you have seen. You can convert an array element of type double to a struct member variable of type int: struct ItemDetails { public string Description; public int ApproxPrice; } //.. double[] Prices = { 25.30, 26.20, 27.40, 30.00 }; ItemDetails id; id.Description = "Hello there."; id.ApproxPrice = (int)(Prices[0] + 0.5); To convert a nullable type to a non-nullable type or another nullable type where data loss may occur, you must use an explicit cast. This is true even when converting between elements with the same basic underlying type — for example, int? to int or float? to float. This is because the nullable type may have the value null, which cannot be represented by the non-nullable type. As long as an explicit cast between two equivalent non-nullable types is possible, so is the explicit cast between nullable types. However, when casting from a nullable type to a non-nullable type and the variable has the value null, an InvalidOperationException is thrown. For example: c07.indd 160 30-01-2014 20:13:15 Type Safety  ❘  161 int? a = null; int b = (int)a; // Will throw exception Using explicit casts and a bit of care and attention, you can convert any instance of a simple value type to almost any other. However, there are limitations on what you can do with explicit type conversions — as far as value types are concerned, you can only convert to and from the numeric and char types and enum types. You cannot directly cast Booleans to any other type or vice versa. If you need to convert between numeric and string, you can use methods provided in the .NET class library. The Object class implements a ToString() method, which has been overridden in all the .NET predefined types and which returns a string representation of the object: int i = 10; string s = i.ToString(); Similarly, if you need to parse a string to retrieve a numeric or Boolean value, you can use the Parse() method supported by all the predefined value types: string s = "100"; int i = int.Parse(s); Console.WriteLine(i + 50); // Add 50 to prove it is really an int Note that Parse() will register an error by throwing an exception if it is unable to convert the string (for example, if you try to convert the string Hello to an integer). Again, exceptions are covered in Chapter 15. Boxing and Unboxing In Chapter 2 you learned that all types — both the simple predefined types such as int and char, and the complex types such as classes and structs — derive from the object type. This means you can treat even literal values as though they are objects: string s = 10.ToString(); However, you also saw that C# data types are divided into value types, which are allocated on the stack, and reference types, which are allocated on the managed heap. How does this square with the capability to call methods on an int, if the int is nothing more than a 4-byte value on the stack? C# achieves this through a bit of magic called boxing. Boxing and its counterpart, unboxing, enable you to convert value types to reference types and then back to value types. We include this in the section on casting because this is essentially what you are doing — you are casting your value to the object type. Boxing is the term used to describe the transformation of a value type to a reference type. Basically, the runtime creates a temporary reference-type box for the object on the heap. This conversion can occur implicitly, as in the preceding example, but you can also perform it explicitly: int myIntNumber = 20; object myObject = myIntNumber; Unboxing is the term used to describe the reverse process, whereby the value of a previously boxed value type is cast back to a value type. We use the term cast here because this has to be done explicitly. The syntax is similar to explicit type conversions already described: int myIntNumber = 20; object myObject = myIntNumber; // Box the int int mySecondNumber = (int)myObject; // Unbox it back into an int A variable can be unboxed only if it has been boxed. If you execute the last line when myObject is not a boxed int, you will get a runtime exception thrown at runtime. One word of warning: When unboxing, you have to be careful that the receiving value variable has enough room to store all the bytes in the value being unboxed. C#’s ints, for example, are c07.indd 161 30-01-2014 20:13:15 162  ❘  CHAPTER 7  Operators and Casts only 32 bits long, so unboxing a long value (64 bits) into an int, as shown here, will result in an InvalidCastException: long myLongNumber = 333333423; object myObject = (object)myLongNumber; int myIntNumber = (int)myObject; Comparing Objects for Equality After discussing operators and briefly touching on the equality operator, it is worth considering for a moment what equality means when dealing with instances of classes and structs. Understanding the mechanics of object equality is essential for programming logical expressions and is important when implementing operator overloads and casts, the topic of the rest of this chapter. The mechanisms of object equality vary depending on whether you are comparing reference types (instances of classes) or value types (the primitive data types, instances of structs, or enums). The following sections present the equality of reference types and value types independently. Comparing Reference Types for Equality You might be surprised to learn that System.Object defines three different methods for comparing objects for equality: ReferenceEquals() and two versions of Equals(). Add to this the comparison operator (==) and you actually have four ways to compare for equality. Some subtle differences exist between the different methods, which are examined next. The ReferenceEquals() Method ReferenceEquals() is a static method that tests whether two references refer to the same instance of a class, specifically whether the two references contain the same address in memory. As a static method, it cannot be overridden, so the System.Object implementation is what you always have. ReferenceEquals() always returns true if supplied with two references that refer to the same object instance, and false otherwise. It does, however, consider null to be equal to null: SomeClass x, y; x = new SomeClass(); y = new SomeClass(); bool B1 = ReferenceEquals(null, null); // returns true bool B2 = ReferenceEquals(null,x); // returns false bool B3 = ReferenceEquals(x, y); // returns false because x and y // point to different objects The Virtual Equals() Method The System.Object implementation of the virtual version of Equals() also works by comparing references. However, because this method is virtual, you can override it in your own classes to compare objects by value. In particular, if you intend instances of your class to be used as keys in a dictionary, you need to override this method to compare values. Otherwise, depending on how you override Object .GetHashCode(), the dictionary class that contains your objects will either not work at all or work very inefficiently. Note that when overriding Equals(), your override should never throw exceptions. Again, that’s because doing so can cause problems for dictionary classes and possibly some other .NET base classes that internally call this method. The Static Equals() Method The static version of Equals() actually does the same thing as the virtual instance version. The difference is that the static version takes two parameters and compares them for equality. This method is able to cope when either of the objects is null; therefore, it provides an extra safeguard against throwing exceptions if c07.indd 162 30-01-2014 20:13:15 Operator Overloading  ❘  163 there is a risk that an object might be null. The static overload first checks whether the references it has been passed are null. If they are both null, it returns true (because null is considered to be equal to null). If just one of them is null, it returns false. If both references actually refer to something, it calls the virtual instance version of Equals(). This means that when you override the instance version of Equals(), the effect is the same as if you were overriding the static version as well. Comparison Operator (==) It is best to think of the comparison operator as an intermediate option between strict value comparison and strict reference comparison. In most cases, writing the following means that you are comparing references: bool b = (x == y); // x, y object references However, it is accepted that there are some classes whose meanings are more intuitive if they are treated as values. In those cases, it is better to override the comparison operator to perform a value comparison. Overriding operators is discussed next, but the obvious example of this is the System.String class for which Microsoft has overridden this operator to compare the contents of the strings rather than their references. Comparing Value Types for Equality When comparing value types for equality, the same principles hold as for reference types: ReferenceEquals() is used to compare references, Equals() is intended for value comparisons, and the comparison operator is viewed as an intermediate case. However, the big difference is that value types need to be boxed to be converted to references so that methods can be executed on them. In addition, Microsoft has already overloaded the instance Equals() method in the System.ValueType class to test equality appropriate to value types. If you call sA.Equals(sB) where sA and sB are instances of some struct, the return value will be true or false, according to whether sA and sB contain the same values in all their fields. On the other hand, no overload of == is available by default for your own structs. Writing (sA == sB) in any expression will result in a compilation error unless you have provided an overload of == in your code for the struct in question. Another point is that ReferenceEquals() always returns false when applied to value types because, to call this method, the value types need to be boxed into objects. Even if you write the following, you will still get the result of false: bool b = ReferenceEquals(v,v); // v is a variable of some value type The reason is because v will be boxed separately when converting each parameter, which means you get different references. Therefore, there really is no reason to call ReferenceEquals() to compare value types because it doesn’t make much sense. Although the default override of Equals() supplied by System.ValueType will almost certainly be adequate for the vast majority of structs that you define, you might want to override it again for your own structs to improve performance. Also, if a value type contains reference types as fields, you might want to override Equals() to provide appropriate semantics for these fields because the default override of Equals() will simply compare their addresses. Operator Overloading This section looks at another type of member that you can define for a class or a struct: the operator overload. Operator overloading is something that will be familiar to C++ developers. However, because the concept is new to both Java and Visual Basic developers, we explain it here. C++ developers will probably prefer to skip ahead to the main operator overloading example. The point of operator overloading is that you do not always just want to call methods or properties on objects. Often, you need to do things like add quantities together, multiply them, or perform logical operations such as comparing objects. Suppose you defined a class that represents a mathematical matrix. In the world c07.indd 163 30-01-2014 20:13:15 164  ❘  CHAPTER 7  Operators and Casts of math, matrices can be added together and multiplied, just like numbers. Therefore, it is quite plausible that you would want to write code like this: Matrix a, b, c; // assume a, b and c have been initialized Matrix d = c * (a + b); By overloading the operators, you can tell the compiler what + and * do when used in conjunction with a Matrix object, enabling you to write code like the preceding. If you were coding in a language that did not support operator overloading, you would have to define methods to perform those operations. The result would certainly be less intuitive and would probably look something like this: Matrix d = c.Multiply(a.Add(b)); With what you have learned so far, operators like + and * have been strictly for use with the predefined data types, and for good reason: The compiler knows what all the common operators mean for those data types. For example, it knows how to add two longs or how to divide one double by another double, and it can generate the appropriate intermediate language code. When you define your own classes or structs, however, you have to tell the compiler everything: what methods are available to call, what fields to store with each instance, and so on. Similarly, if you want to use operators with your own types, you have to tell the compiler what the relevant operators mean in the context of that class. You do that by defining overloads for the operators. The other thing to stress is that overloading is not just concerned with arithmetic operators. You also need to consider the comparison operators, ==, <, >, !=, >=, and <=. Take the statement if (a==b). For classes, this statement will, by default, compare the references a and b. It tests whether the references point to the same location in memory, rather than checking whether the instances actually contain the same data. For the string class, this behavior is overridden so that comparing strings really does compare the contents of each string. You might want to do the same for your own classes. For structs, the == operator does not do anything at all by default. Trying to compare two structs to determine whether they are equal produces a compilation error unless you explicitly overload == to tell the compiler how to perform the comparison. In many situations, being able to overload operators enables you to generate more readable and intuitive code, including the following: ➤➤ Almost any mathematical object such as coordinates, vectors, matrices, tensors, functions, and so on. If you are writing a program that does some mathematical or physical modeling, you will almost certainly use classes representing these objects. ➤➤ Graphics programs that use mathematical or coordinate-related objects when calculating positions on-screen. ➤➤ A class that represents an amount of money (for example, in a financial program). ➤➤ A word processing or text analysis program that uses classes representing sentences, clauses, and so on. You might want to use operators to combine sentences (a more sophisticated version of concatenation for strings). However, there are also many types for which operator overloading is not relevant. Using operator overloading inappropriately will make any code that uses your types far more difficult to understand. For example, multiplying two DateTime objects does not make any sense conceptually. How Operators Work To understand how to overload operators, it’s quite useful to think about what happens when the compiler encounters an operator. Using the addition operator (+) as an example, suppose that the compiler processes the following lines of code: int myInteger = 3; uint myUnsignedInt = 2; double myDouble = 4.0; long myLong = myInteger + myUnsignedInt; double myOtherDouble = myDouble + myInteger; c07.indd 164 30-01-2014 20:13:15 Operator Overloading  ❘  165 Now consider what happens when the compiler encounters this line: long myLong = myInteger + myUnsignedInt; The compiler identifies that it needs to add two integers and assign the result to a long. However, the expression myInteger + myUnsignedInt is really just an intuitive and convenient syntax for calling a method that adds two numbers. The method takes two parameters, myInteger and myUnsignedInt, and returns their sum. Therefore, the compiler does the same thing it does for any method call: It looks for the best matching overload of the addition operator based on the parameter types — in this case, one that takes two integers. As with normal overloaded methods, the desired return type does not influence the compiler’s choice as to which version of a method it calls. As it happens, the overload called in the example takes two int parameters and returns an int; this return value is subsequently converted to a long. The next line causes the compiler to use a different overload of the addition operator: double myOtherDouble = myDouble + myInteger; In this instance, the parameters are a double and an int, but there is no overload of the addition operator that takes this combination of parameters. Instead, the compiler identifies the best matching overload of the addition operator as being the version that takes two doubles as its parameters, and it implicitly casts the int to a double. Adding two doubles requires a different process from adding two integers. Floating-point numbers are stored as a mantissa and an exponent. Adding them involves bit-shifting the mantissa of one of the doubles so that the two exponents have the same value, adding the mantissas, then shifting the mantissa of the result and adjusting its exponent to maintain the highest possible accuracy in the answer. Now you are in a position to see what happens if the compiler finds something like this: Vector vect1, vect2, vect3; // initialize vect1 and vect2 vect3 = vect1 + vect2; vect1 = vect1*2; Here, Vector is the struct, which is defined in the following section. The compiler sees that it needs to add two Vector instances, vect1 and vect2, together. It looks for an overload of the addition operator, which takes two Vector instances as its parameters. If the compiler finds an appropriate overload, it calls up the implementation of that operator. If it cannot find one, it checks whether there is any other overload for + that it can use as a best match — perhaps something with two parameters of other data types that can be implicitly converted to Vector instances. If the compiler cannot find a suitable overload, it raises a compilation error, just as it would if it could not find an appropriate overload for any other method call. Operator Overloading Example: The Vector Struct This section demonstrates operator overloading through developing a struct named Vector that represents a three-dimensional mathematical vector. Don’t worry if mathematics is not your strong point — the vector example is very simple. As far as you are concerned here, a 3D vector is just a set of three numbers (doubles) that tell you how far something is moving. The variables representing the numbers are called x, y, and z: the x tells you how far something moves east, y tells you how far it moves north, and z tells you how far it moves upward (in height). Combine the three numbers and you get the total movement. For example, if x=3.0, y=3.0, and z=1.0 (which you would normally write as (3.0, 3.0, 1.0), you’re moving 3 units east, 3 units north, and rising upward by 1 unit. You can add or multiply vectors by other vectors or by numbers. Incidentally, in this context, we use the term scalar, which is math-speak for a simple number — in C# terms that is just a double. The significance of addition should be clear. If you move first by the vector (3.0, 3.0, 1.0) then you move by the vector (2.0, -4.0, -4.0), the total amount you have moved can be determined by adding the two vectors. Adding vectors means adding each component individually, so you get (5.0, -1.0, -3.0). In this context, mathematicians write c=a+b, where a and b are the vectors and c is the resulting vector. You want to be able to use the Vector struct the same way. c07.indd 165 30-01-2014 20:13:16 166  ❘  CHAPTER 7  Operators and Casts Note  The fact that this example is developed as a struct rather than a class is not significant. Operator overloading works in the same way for both structs and classes. Following is the definition for Vector — containing the member fields, constructors, a ToString() override so you can easily view the contents of a Vector, and, finally, that operator overload: namespace Wrox.ProCSharp.OOCSharp { struct Vector { public double x, y, z; public Vector(double x, double y, double z) { this.x = x; this.y = y; this.z = z; } public Vector(Vector rhs) { x = rhs.x; y = rhs.y; z = rhs.z; } public override string ToString() { return "( " + x + ", " + y + ", " + z + " )"; } This example has two constructors that require specifying the initial value of the vector, either by passing in the values of each component or by supplying another Vector whose value can be copied. Constructors like the second one, that takes a single Vector argument, are often termed copy constructors because they effectively enable you to initialize a class or struct instance by copying another instance. Note that to keep things simple, the fields are left as public. We could have made them private and written corresponding properties to access them, but it would not make any difference to the example, other than to make the code longer. Here is the interesting part of the Vector struct — the operator overload that provides support for the addition operator: public static Vector operator + (Vector lhs, Vector rhs) { Vector result = new Vector(lhs); result.x += rhs.x; result.y += rhs.y; result.z += rhs.z; return result; } } } The operator overload is declared in much the same way as a method, except that the operator keyword tells the compiler it is actually an operator overload you are defining. The operator keyword is followed by the actual symbol for the relevant operator, in this case the addition operator (+). The return type is whatever type you get when you use this operator. Adding two vectors results in a vector; therefore, the return type is also a Vector. For this particular override of the addition operator, the return type is the same as the containing class, but that is not necessarily the case, as you will see later in this example. The two parameters are the things you are operating on. For binary operators (those that take two parameters), such as the c07.indd 166 30-01-2014 20:13:16 Operator Overloading  ❘  167 addition and subtraction operators, the first parameter is the value on the left of the operator, and the second parameter is the value on the right. Note  It is conventional to name your left-hand parameters lhs (for left-hand side) and your right-hand parameters rhs (for right-hand side). C# requires that all operator overloads be declared as public and static, which means they are associated with their class or struct, not with a particular instance. Because of this, the body of the operator overload has no access to non-static class members or the this identifier. This is fine because the parameters provide all the input data the operator needs to know to perform its task. Now that you understand the syntax for the addition operator declaration, examine what happens inside the operator: { Vector result = new Vector(lhs); result.x += rhs.x; result.y += rhs.y; result.z += rhs.z; return result; } This part of the code is exactly the same as if you were declaring a method, and you should easily be able to convince yourself that this will return a vector containing the sum of lhs and rhs as defined. You simply add the members x, y, and z together individually. Now all you need to do is write some simple code to test the Vector struct: static void Main() { Vector vect1, vect2, vect3; vect1 = new Vector(3.0, 3.0, 1.0); vect2 = new Vector(2.0, -4.0, -4.0); vect3 = vect1 + vect2; Console.WriteLine("vect1 = " + vect1.ToString()); Console.WriteLine("vect2 = " + vect2.ToString()); Console.WriteLine("vect3 = " + vect3.ToString()); } Saving this code as Vectors.cs and compiling and running it returns this result: vect1 = ( 3, 3, 1 ) vect2 = ( 2, -4, -4 ) vect3 = ( 5, -1, -3 ) Adding More Overloads In addition to adding vectors, you can multiply and subtract them and compare their values. In this section, you develop the Vector example further by adding a few more operator overloads. You won’t develop the complete set that you’d probably need for a fully functional Vector type, but just enough to demonstrate some other aspects of operator overloading. First, you’ll overload the multiplication operator to support multiplying vectors by a scalar and multiplying vectors by another vector. Multiplying a vector by a scalar simply means multiplying each component individually by the scalar: for example, 2 * (1.0, 2.5, 2.0) returns (2.0, 5.0, 4.0). The relevant operator overload looks similar to this: public static Vector operator * (double lhs, Vector rhs) { c07.indd 167 30-01-2014 20:13:16 168  ❘  CHAPTER 7  Operators and Casts return new Vector(lhs * rhs.x, lhs * rhs.y, lhs * rhs.z); } This by itself, however, is not sufficient. If a and b are declared as type Vector, you can write code like this: b = 2 * a; The compiler will implicitly convert the integer 2 to a double to match the operator overload signature. However, code like the following will not compile: b = a * 2; The point is that the compiler treats operator overloads exactly like method overloads. It examines all the available overloads of a given operator to find the best match. The preceding statement requires the first parameter to be a Vector and the second parameter to be an integer, or something to which an integer can be implicitly converted. You have not provided such an overload. The compiler cannot start swapping the order of parameters, so the fact that you’ve provided an overload that takes a double followed by a Vector is not sufficient. You need to explicitly define an overload that takes a Vector followed by a double as well. There are two possible ways of implementing this. The first way involves breaking down the vector multiplication operation in the same way that you have done for all operators so far: public static Vector operator * (Vector lhs, double rhs) { return new Vector(rhs * lhs.x, rhs * lhs.y, rhs *lhs.z); } Given that you have already written code to implement essentially the same operation, however, you might prefer to reuse that code by writing the following: public static Vector operator * (Vector lhs, double rhs) { return rhs * lhs; } This code works by effectively telling the compiler that when it sees a multiplication of a Vector by a double, it can simply reverse the parameters and call the other operator overload. The sample code for this chapter uses the second version, because it looks neater and illustrates the idea in action. This version also makes the code more maintainable because it saves duplicating the code to perform the multiplication in two separate overloads. Next, you need to overload the multiplication operator to support vector multiplication. Mathematics provides a couple of ways to multiply vectors, but the one we are interested in here is known as the dot product or inner product, which actually returns a scalar as a result. That’s the reason for this example, to demonstrate that arithmetic operators don’t have to return the same type as the class in which they are defined. In mathematical terms, if you have two vectors (x, y, z) and (X, Y, Z), then the inner product is defined to be the value of x*X + y*Y + z*Z. That might look like a strange way to multiply two things together, but it is actually very useful because it can be used to calculate various other quantities. If you ever write code that displays complex 3D graphics, such as using Direct3D or DirectDraw, you will almost certainly find that your code needs to work out inner products of vectors quite often as an intermediate step in calculating where to place objects on the screen. What concerns us here is that we want users of your Vector to be able to write double X = a*b to calculate the inner product of two Vector objects (a and b). The relevant overload looks like this: public static double operator * (Vector lhs, Vector rhs) { return lhs.x * rhs.x + lhs.y * rhs.y + lhs.z * rhs.z; } Now that you understand the arithmetic operators, you can confirm that they work using a simple test method: static void Main() { // stuff to demonstrate arithmetic operations Vector vect1, vect2, vect3; vect1 = new Vector(1.0, 1.5, 2.0); c07.indd 168 30-01-2014 20:13:16 Operator Overloading  ❘  169 vect2 = new Vector(0.0, 0.0, -10.0); vect3 = vect1 + vect2; Console.WriteLine("vect1 = " + vect1); Console.WriteLine("vect2 = " + vect2); Console.WriteLine("vect3 = vect1 + vect2 = " + vect3); Console.WriteLine("2*vect3 = " + 2*vect3); vect3 += vect2; Console.WriteLine("vect3+=vect2 gives " + vect3); vect3 = vect1*2; Console.WriteLine("Setting vect3=vect1*2 gives " + vect3); double dot = vect1*vect3; Console.WriteLine("vect1*vect3 = " + dot); } Running this code (Vectors2.cs) produces the following result: Vectors2 vect1 = ( 1, 1.5, 2 ) vect2 = ( 0, 0, -10 ) vect3 = vect1 + vect2 = ( 1, 1.5, -8 ) 2*vect3 = ( 2, 3, -16 ) vect3+=vect2 gives ( 1, 1.5, -18 ) Setting vect3=vect1*2 gives ( 2, 3, 4 ) vect1*vect3 = 14.5 This shows that the operator overloads have given the correct results; but if you look at the test code closely, you might be surprised to notice that it actually used an operator that wasn’t overloaded — the addition assignment operator, +=: vect3 += vect2; Console.WriteLine("vect3 += vect2 gives " + vect3); Although += normally counts as a single operator, it can be broken down into two steps: the addition and the assignment. Unlike the C++ language, C# does not allow you to overload the = operator; but if you overload +, the compiler will automatically use your overload of + to work out how to perform a += operation. The same principle works for all the assignment operators, such as -=, *=, /=, &=, and so on. Overloading the Comparison Operators As shown earlier in the section “Operators,” C# has six comparison operators, and they are paired as follows: ➤➤ == and != ➤➤ > and < ➤➤ >= and <= The C# language requires that you overload these operators in pairs. That is, if you overload ==, you must overload != too; otherwise, you get a compiler error. In addition, the comparison operators must return a bool. This is the fundamental difference between these operators and the arithmetic operators. The result of adding or subtracting two quantities, for example, can theoretically be any type depending on the quantities. You have already seen that multiplying two Vector objects can be implemented to give a scalar. Another example involves the .NET base class System.DateTime. It’s possible to subtract two DateTime instances, but the result is not a DateTime; instead it is a System.TimeSpan instance. By contrast, it doesn’t really make much sense for a comparison to return anything other than a bool. c07.indd 169 30-01-2014 20:13:16 170  ❘  CHAPTER 7  Operators and Casts Note  If you overload == and !=, you must also override the Equals() and GetHashCode() methods inherited from System.Object; otherwise, you’ll get a compiler warning. The reasoning is that the Equals() method should implement the same kind of equality logic as the == operator. Apart from these differences, overloading the comparison operators follows the same principles as overloading the arithmetic operators. However, comparing quantities isn’t always as simple as you might think. For example, if you simply compare two object references, you will compare the memory address where the objects are stored. This is rarely the desired behavior of a comparison operator, so you must code the operator to compare the value of the objects and return the appropriate Boolean response. The following example overrides the == and != operators for the Vector struct. Here is the implementation of ==: public static bool operator == (Vector lhs, Vector rhs) { if (lhs.x == rhs.x && lhs.y == rhs.y && lhs.z == rhs.z) return true; else return false; } This approach simply compares two Vector objects for equality based on the values of their components. For most structs, that is probably what you will want to do, though in some cases you may need to think carefully about what you mean by equality. For example, if there are embedded classes, should you simply compare whether the references point to the same object (shallow comparison) or whether the values of the objects are the same (deep comparison)? With a shallow comparison, the objects point to the same point in memory, whereas deep comparisons work with values and properties of the object to deem equality. You want to perform equality checks depending on the depth to help you decide what you want to verify. Note  Don’t be tempted to overload the comparison operator by calling the instance version of the Equals() method inherited from System.Object. If you do and then an attempt is made to evaluate (objA == objB), when objA happens to be null, you will get an exception, as the .NET runtime tries to evaluate null.Equals(objB). Working the other way around (overriding Equals() to call the comparison operator) should be safe. You also need to override the != operator. Here is the simple way to do this: public static bool operator != (Vector lhs, Vector rhs) { return ! (lhs == rhs); } As usual, you should quickly confirm that your override works with some test code. This time you’ll define three Vector objects and compare them: static void Main() { Vector vect1, vect2, vect3; vect1 = new Vector(3.0, 3.0, -10.0); vect2 = new Vector(3.0, 3.0, -10.0); vect3 = new Vector(2.0, 3.0, 6.0); Console.WriteLine("vect1==vect2 returns " + (vect1==vect2)); Console.WriteLine("vect1==vect3 returns " + (vect1==vect3)); c07.indd 170 30-01-2014 20:13:16 Operator Overloading  ❘  171 Console.WriteLine("vect2==vect3 returns " + (vect2==vect3)); Console.WriteLine(); Console.WriteLine("vect1!=vect2 returns " + (vect1!=vect2)); Console.WriteLine("vect1!=vect3 returns " + (vect1!=vect3)); Console.WriteLine("vect2!=vect3 returns " + (vect2!=vect3)); } Compiling this code (the Vectors3.cs sample in the code download) generates the following compiler warning because you haven’t overridden Equals() for your Vector. For our purposes here, that doesn’t, so we will ignore it: Microsoft (R) Visual C# 2010 Compiler version 4.0.21006.1 for Microsoft (R) .NET Framework version 4.0 Copyright (C) Microsoft Corporation. All rights reserved. Vectors3.cs(5,11): warning CS0660: 'Wrox.ProCSharp.OOCSharp.Vector' defines operator == or operator != but does not override Object.Equals(object o) Vectors3.cs(5,11): warning CS0661: 'Wrox.ProCSharp.OOCSharp.Vector' defines operator == or operator != but does not override Object.GetHashCode() Running the example produces these results at the command line: Vectors3 vect1==vect2 returns True vect1==vect3 returns False vect2==vect3 returns False vect1!=vect2 returns False vect1!=vect3 returns True vect2!=vect3 returns True Which Operators Can You Overload? It is not possible to overload all the available operators. The operators that you can overload are listed in the following table: Category Operators Restrictions Arithmetic binary +, *, /, -, % None Arithmetic unary +, -, ++, -- None Bitwise binary &, |, ^, <<, >> None Bitwise unary !, ~true, false The true and false operators must be overloaded as a pair. Comparison ==, !=,>=, <=>, <, Comparison operators must be overloaded in pairs. Assignment +=, -=, *=, /=, >>=, <<=, %=, &=, |=, ^= You cannot explicitly overload these operators; they are overridden implicitly when you override the individual operators such as +, -, %, and so on. Index [] You cannot overload the index operator directly. The indexer member type, discussed in Chapter 2, allows you to support the index operator on your classes and structs. Cast () You cannot overload the cast operator directly. User- defined casts (discussed next) allow you to define custom cast behavior. c07.indd 171 30-01-2014 20:13:17 172  ❘  CHAPTER 7  Operators and Casts User-Defined Casts Earlier in this chapter (see the “Explicit Conversions” section), you learned that you can convert values between predefined data types through a process of casting. You also saw that C# allows two different types of casts: implicit and explicit. This section looks at these types of casts. For an explicit cast, you explicitly mark the cast in your code by including the destination data type inside parentheses: int I = 3; long l = I; // implicit short s = (short)I; // explicit For the predefined data types, explicit casts are required where there is a risk that the cast might fail or some data might be lost. The following are some examples: ➤➤ When converting from an int to a short, the short might not be large enough to hold the value of the int. ➤➤ When converting from signed to unsigned data types, incorrect results are returned if the signed variable holds a negative value. ➤➤ When converting from floating-point to integer data types, the fractional part of the number will be lost. ➤➤ When converting from a nullable type to a non-nullable type, a value of null causes an exception. By making the cast explicit in your code, C# forces you to affirm that you understand there is a risk of data loss, and therefore presumably you have written your code to take this into account. Because C# allows you to define your own data types (structs and classes), it follows that you need the facility to support casts to and from those data types. The mechanism is to define a cast as a member operator of one of the relevant classes. Your cast operator must be marked as either implicit or explicit to indicate how you are intending it to be used. The expectation is that you follow the same guidelines as for the predefined casts: if you know that the cast is always safe regardless of the value held by the source variable, then you define it as implicit. Conversely, if you know there is a risk of something going wrong for certain values — perhaps some loss of data or an exception being thrown — then you should define the cast as explicit. Note  You should define any custom casts you write as explicit if there are any source data values for which the cast will fail or if there is any risk of an exception being thrown. The syntax for defining a cast is similar to that for overloading operators discussed earlier in this chapter. This is not a coincidence — a cast is regarded as an operator whose effect is to convert from the source type to the destination type. To illustrate the syntax, the following is taken from an example struct named Currency, which is introduced later in this section: public static implicit operator float (Currency value) { // processing } The return type of the operator defines the target type of the cast operation, and the single parameter is the source object for the conversion. The cast defined here allows you to implicitly convert the value of a Currency into a float. Note that if a conversion has been declared as implicit, the compiler permits its use either implicitly or explicitly. If it has been declared as explicit, the compiler only permits it to be used explicitly. In common with other operator overloads, casts must be declared as both public and static. c07.indd 172 30-01-2014 20:13:17 User-Defined Casts  ❘  173 Note  C++ developers will notice that this is different from C++, in which casts are instance members of classes. Implementing User-Defined Casts This section illustrates the use of implicit and explicit user-defined casts in an example called SimpleCurrency (which, as usual, is available in the code download). In this example, you define a struct, Currency, which holds a positive USD ($) monetary value. C# provides the decimal type for this purpose, but it is possible you will still want to write your own struct or class to represent monetary values if you need to perform sophisticated financial processing and therefore want to implement specific methods on such a class. Note  The syntax for casting is the same for structs and classes. This example happens to be for a struct, but it would work just as well if you declared Currency as a class. Initially, the definition of the Currency struct is as follows: struct Currency { public uint Dollars; public ushort Cents; public Currency(uint dollars, ushort cents) { this.Dollars = dollars; this.Cents = cents; } public override string ToString() { return string.Format("${0}.{1,-2:00}", Dollars,Cents); } } The use of unsigned data types for the Dollar and Cents fields ensures that a Currency instance can hold only positive values. It is restricted this way to illustrate some points about explicit casts later. You might want to use a class like this to hold, for example, salary information for company employees (people’s salaries tend not to be negative!). To keep the class simple, the fields are public, but usually you would make them private and define corresponding properties for the dollars and cents. Start by assuming that you want to be able to convert Currency instances to float values, where the integer part of the float represents the dollars. In other words, you want to be able to write code like this: Currency balance = new Currency(10,50); float f = balance; // We want f to be set to 10.5 To be able to do this, you need to define a cast. Hence, you add the following to your Currency definition: public static implicit operator float (Currency value) { return value.Dollars + (value.Cents/100.0f); } The preceding cast is implicit. It is a sensible choice in this case because, as it should be clear from the definition of Currency, any value that can be stored in the currency can also be stored in a float. There is no way that anything should ever go wrong in this cast. c07.indd 173 30-01-2014 20:13:17 174  ❘  CHAPTER 7  Operators and Casts Note  There is a slight cheat here: in fact, when converting a uint to a float, there can be a loss in precision, but Microsoft has deemed this error sufficiently marginal to count the uint-to-float cast as implicit. However, if you have a float that you would like to be converted to a Currency, the conversion is not guaranteed to work. A float can store negative values, which Currency instances can’t, and a float can store numbers of a far higher magnitude than can be stored in the (uint) Dollar field of Currency. Therefore, if a float contains an inappropriate value, converting it to a Currency could give unpredictable results. Because of this risk, the conversion from float to Currency should be defined as explicit. Here is the first attempt, which will not return quite the correct results, but it is instructive to examine why: public static explicit operator Currency (float value) { uint dollars = (uint)value; ushort cents = (ushort)((value-dollars)*100); return new Currency(dollars, cents); } The following code will now successfully compile: float amount = 45.63f; Currency amount2 = (Currency)amount; However, the following code, if you tried it, would generate a compilation error, because it attempts to use an explicit cast implicitly: float amount = 45.63f; Currency amount2 = amount; // wrong By making the cast explicit, you warn the developer to be careful because data loss might occur. However, as you will soon see, this is not how you want your Currency struct to behave. Try writing a test harness and running the sample. Here is the Main() method, which instantiates a Currency struct and attempts a few conversions. At the start of this code, you write out the value of balance in two different ways (this will be needed to illustrate something later in the example): static void Main() { try { Currency balance = new Currency(50,35); Console.WriteLine(balance); Console.WriteLine("balance is " + balance); Console.WriteLine("balance is (using ToString()) " + balance.ToString()); float balance2= balance; Console.WriteLine("After converting to float, = " + balance2); balance = (Currency) balance2; Console.WriteLine("After converting back to Currency, = " + balance); Console.WriteLine("Now attempt to convert out of range value of " + "-$50.50 to a Currency:"); checked { balance = (Currency) (-50.50); Console.WriteLine("Result is " + balance.ToString()); } c07.indd 174 30-01-2014 20:13:17 User-Defined Casts  ❘  175 } catch(Exception e) { Console.WriteLine("Exception occurred: " + e.Message); } } Notice that the entire code is placed in a try block to catch any exceptions that occur during your casts. In addition, the lines that test converting an out-of-range value to Currency are placed in a checked block in an attempt to trap negative values. Running this code produces the following output: SimpleCurrency 50.35 Balance is $50.35 Balance is (using ToString()) $50.35 After converting to float, = 50.35 After converting back to Currency, = $50.34 Now attempt to convert out of range value of -$100.00 to a Currency: Result is $4294967246.00 This output shows that the code did not quite work as expected. First, converting back from float to Currency gave a wrong result of $50.34 instead of $50.35. Second, no exception was generated when you tried to convert an obviously out-of-range value. The first problem is caused by rounding errors. If a cast is used to convert from a float to a uint, the computer will truncate the number rather than round it. The computer stores numbers in binary rather than decimal, and the fraction 0.35 cannot be exactly represented as a binary fraction (just as 1/3 cannot be represented exactly as a decimal fraction; it comes out as 0.3333 recurring). The computer ends up storing a value very slightly lower than 0.35 that can be represented exactly in binary format. Multiply by 100 and you get a number fractionally less than 35, which is truncated to 34 cents. Clearly, in this situation, such errors caused by truncation are serious, and the way to avoid them is to ensure that some intelligent rounding is performed in numerical conversions instead. Luckily, Microsoft has written a class that does this: System.Convert. The System.Convert object contains a large number of static methods to perform various numerical conversions, and the one that we want is Convert.ToUInt16(). Note that the extra care taken by the System.Convert methods does come at a performance cost. You should use them only when necessary. Let’s examine the second problem — why the expected overflow exception wasn’t thrown. The issue here is this: The place where the overflow really occurs isn’t actually in the Main() routine at all — it is inside the code for the cast operator, which is called from the Main() method. The code in this method was not marked as checked. The solution is to ensure that the cast itself is computed in a checked context too. With both this change and the fix for the first problem, the revised code for the conversion looks like the following: public static explicit operator Currency (float value) { checked { uint dollars = (uint)value; ushort cents = Convert.ToUInt16((value-dollars)*100); return new Currency(dollars, cents); } } Note that you use Convert.ToUInt16() to calculate the cents, as described earlier, but you do not use it for calculating the dollar part of the amount. System.Convert is not needed when calculating the dollar amount because truncating the float value is what you want there. c07.indd 175 30-01-2014 20:13:17 176  ❘  CHAPTER 7  Operators and Casts Note  The System.Convert methods also carry out their own overflow checking. Hence, for the particular case we are considering, there is no need to place the call to Convert.ToUInt16() inside the checked context. The checked context is still required, however, for the explicit casting of value to dollars. You won’t see a new set of results with this new checked cast just yet because you have some more modifications to make to the SimpleCurrency example later in this section. Note  If you are defining a cast that will be used very often, and for which performance is at an absolute premium, you may prefer not to do any error checking. That is also a legitimate solution, provided that the behavior of your cast and the lack of error checking are very clearly documented. Casts Between Classes The Currency example involves only classes that convert to or from float — one of the predefined data types. However, it is not necessary to involve any of the simple data types. It is perfectly legitimate to define casts to convert between instances of different structs or classes that you have defined. You need to be aware of a couple of restrictions, however: ➤➤ You cannot define a cast if one of the classes is derived from the other (these types of casts already exist, as you will see). ➤➤ The cast must be defined inside the definition of either the source or the destination data type. To illustrate these requirements, suppose that you have the class hierarchy shown in Figure 7-1. In other words, classes C and D are indirectly derived from A. In this case, the only legitimate user-defined cast between A, B, C, or D would be to convert between classes C and D, because these classes are not derived from each other. The code to do so might look like the following (assuming you want the casts to be explicit, which is usually the case when defining casts between user-defined classes): public static explicit operator D(C value) { // and so on } public static explicit operator C(D value) { // and so on } For each of these casts, you can choose where you place the definitions — inside the class definition of C or inside the class definition of D, but not anywhere else. C# requires you to put the definition of a cast inside either the source class (or struct) or the destination class (or struct). A side effect of this is that you cannot define a cast between two classes unless you have access to edit the source code for at least one of them. This is sensible because it prevents third parties from introducing casts into your classes. After you have defined a cast inside one of the classes, you cannot also define the same cast inside the other class. Obviously, there should be only one cast for each conversion; otherwise, the compiler would not know which one to use. Casts Between Base and Derived Classes To see how these casts work, start by considering the case in which both the source and the destination are reference types, and consider two classes, MyBase and MyDerived, where MyDerived is derived directly or indirectly from MyBase. System Object A B C D Figure 7-1 c07.indd 176 30-01-2014 20:13:19 User-Defined Casts  ❘  177 First, from MyDerived to MyBase, it is always possible (assuming the constructors are available) to write this: MyDerived derivedObject = new MyDerived(); MyBase baseCopy = derivedObject; Here, you are casting implicitly from MyDerived to MyBase. This works because of the rule that any reference to a type MyBase is allowed to refer to objects of class MyBase or anything derived from MyBase. In OO programming, instances of a derived class are, in a real sense, instances of the base class, plus something extra. All the functions and fields defined on the base class are defined in the derived class too. Alternatively, you can write this: MyBase derivedObject = new MyDerived(); MyBase baseObject = new MyBase(); MyDerived derivedCopy1 = (MyDerived) derivedObject; // OK MyDerived derivedCopy2 = (MyDerived) baseObject; // Throws exception This code is perfectly legal C# (in a syntactic sense, that is) and illustrates casting from a base class to a derived class. However, the final statement will throw an exception when executed. When you perform the cast, the object being referred to is examined. Because a base class reference can, in principle, refer to a derived class instance, it is possible that this object is actually an instance of the derived class that you are attempting to cast to. If that is the case, the cast succeeds, and the derived reference is set to refer to the object. If, however, the object in question is not an instance of the derived class (or of any class derived from it), the cast fails and an exception is thrown. Notice that the casts that the compiler has supplied, which convert between base and derived class, do not actually do any data conversion on the object in question. All they do is set the new reference to refer to the object if it is legal for that conversion to occur. To that extent, these casts are very different in nature from the ones that you normally define yourself. For example, in the SimpleCurrency example earlier, you defined casts that convert between a Currency struct and a float. In the float-to-Currency cast, you actually instantiated a new Currency struct and initialized it with the required values. The predefined casts between base and derived classes do not do this. If you want to convert a MyBase instance into a real MyDerived object with values based on the contents of the MyBase instance, you cannot use the cast syntax to do this. The most sensible option is usually to define a derived class constructor that takes a base class instance as a parameter, and have this constructor perform the relevant initializations: class DerivedClass: BaseClass { public DerivedClass(BaseClass rhs) { // initialize object from the Base instance } // etc. Boxing and Unboxing Casts The previous discussion focused on casting between base and derived classes where both participants were reference types. Similar principles apply when casting value types, although in this case it is not possible to simply copy references — some copying of data must occur. It is not, of course, possible to derive from structs or primitive value types. Casting between base and derived structs invariably means casting between a primitive type or a struct and System.Object. (Theoretically, it is possible to cast between a struct and System.ValueType, though it is hard to see why you would want to do this.) The cast from any struct (or primitive type) to object is always available as an implicit cast — because it is a cast from a derived type to a base type — and is just the familiar process of boxing. For example, using the Currency struct: Currency balance = new Currency(40,0); object baseCopy = balance; c07.indd 177 30-01-2014 20:13:19 178  ❘  CHAPTER 7  Operators and Casts When this implicit cast is executed, the contents of balance are copied onto the heap into a boxed object, and the baseCopy object reference is set to this object. What actually happens behind the scenes is this: When you originally defined the Currency struct, the .NET Framework implicitly supplied another (hidden) class, a boxed Currency class, which contains all the same fields as the Currency struct but is a reference type, stored on the heap. This happens whenever you define a value type, whether it is a struct or an enum, and similar boxed reference types exist corresponding to all the primitive value types of int, double, uint, and so on. It is not possible, or necessary, to gain direct programmatic access to any of these boxed classes in source code, but they are the objects that are working behind the scenes whenever a value type is cast to object. When you implicitly cast Currency to object, a boxed Currency instance is instantiated and initialized with all the data from the Currency struct. In the preceding code, it is this boxed Currency instance to which baseCopy refers. By these means, it is possible for casting from derived to base type to work syntactically in the same way for value types as for reference types. Casting the other way is known as unboxing. Like casting between a base reference type and a derived reference type, it is an explicit cast because an exception will be thrown if the object being cast is not of the correct type: object derivedObject = new Currency(40,0); object baseObject = new object(); Currency derivedCopy1 = (Currency)derivedObject; // OK Currency derivedCopy2 = (Currency)baseObject; // Exception thrown This code works in a way similar to the code presented earlier for reference types. Casting derivedObject to Currency works fine because derivedObject actually refers to a boxed Currency instance — the cast is performed by copying the fields out of the boxed Currency object into a new Currency struct. The second cast fails because baseObject does not refer to a boxed Currency object. When using boxing and unboxing, it is important to understand that both processes actually copy the data into the new boxed or unboxed object. Hence, manipulations on the boxed object, for example, will not affect the contents of the original value type. Multiple Casting One thing you will have to watch for when you are defining casts is that if the C# compiler is presented with a situation in which no direct cast is available to perform a requested conversion, it will attempt to find a way of combining casts to do the conversion. For example, with the Currency struct, suppose the compiler encounters a few lines of code like this: Currency balance = new Currency(10,50); long amount = (long)balance; double amountD = balance; You first initialize a Currency instance, and then you attempt to convert it to a long. The trouble is that you haven’t defined the cast to do that. However, this code still compiles successfully. What will happen is that the compiler will realize that you have defined an implicit cast to get from Currency to float, and the compiler already knows how to explicitly cast a float to a long. Hence, it will compile that line of code into IL code that converts balance first to a float, and then converts that result to a long. The same thing happens in the final line of the code, when you convert balance to a double. However, because the cast from Currency to float and the predefined cast from float to double are both implicit, you can write this conversion in your code as an implicit cast. If you prefer, you could also specify the casting route explicitly: Currency balance = new Currency(10,50); long amount = (long)(float)balance; double amountD = (double)(float)balance; However, in most cases, this would be seen as needlessly complicating your code. The following code, by contrast, produces a compilation error: Currency balance = new Currency(10,50); long amount = balance; c07.indd 178 30-01-2014 20:13:19 User-Defined Casts  ❘  179 The reason is that the best match for the conversion that the compiler can find is still to convert first to float and then to long. The conversion from float to long needs to be specified explicitly, though. Not all of this by itself should give you too much trouble. The rules are, after all, fairly intuitive and designed to prevent any data loss from occurring without the developer knowing about it. However, the problem is that if you are not careful when you define your casts, it is possible for the compiler to select a path that leads to unexpected results. For example, suppose that it occurs to someone else in the group writing the Currency struct that it would be useful to be able to convert a uint containing the total number of cents in an amount into a Currency (cents, not dollars, because the idea is not to lose the fractions of a dollar). Therefore, this cast might be written to try to achieve this: public static implicit operator Currency (uint value) { return new Currency(value/100u, (ushort)(value%100)); } // Do not do this! Note the u after the first 100 in this code to ensure that value/100u is interpreted as a uint. If you had written value/100, the compiler would have interpreted this as an int, not a uint. The comment Do not do this! is clearly noted in this code, and here is why: The following code snippet merely converts a uint containing 350 into a Currency and back again; but what do you think bal2 will contain after executing this? uint bal = 350; Currency balance = bal; uint bal2 = (uint)balance; The answer is not 350 but 3! Moreover, it all follows logically. You convert 350 implicitly to a Currency, giving the result balance.Dollars = 3, balance.Cents = 50. Then the compiler does its usual figuring out of the best path for the conversion back. Balance ends up being implicitly converted to a float (value 3.5), and this is converted explicitly to a uint with value 3. Of course, other instances exist in which converting to another data type and back again causes data loss. For example, converting a float containing 5.8 to an int and back to a float again will lose the fractional part, giving you a result of 5, but there is a slight difference in principle between losing the fractional part of a number and dividing an integer by more than 100. Currency has suddenly become a rather dangerous class that does strange things to integers! The problem is that there is a conflict between how your casts interpret integers. The casts between Currency and float interpret an integer value of 1 as corresponding to one dollar, but the latest uint-to-Currency cast interprets this value as one cent. This is an example of very poor design. If you want your classes to be easy to use, you should ensure that all your casts behave in a way that is mutually compatible, in the sense that they intuitively give the same results. In this case, the solution is obviously to rewrite the uint-to-Currency cast so that it interprets an integer value of 1 as one dollar: public static implicit operator Currency (uint value) { return new Currency(value, 0); } Incidentally, you might wonder whether this new cast is necessary at all. The answer is that it could be useful. Without this cast, the only way for the compiler to carry out a uint-to-Currency conversion would be via a float. Converting directly is a lot more efficient in this case, so having this extra cast provides performance benefits, though you need to ensure that it provides the same result as via a float, which you have now done. In other situations, you may also find that separately defining casts for different predefined data types enables more conversions to be implicit rather than explicit, though that is not the case here. A good test of whether your casts are compatible is to ask whether a conversion will give the same results (other than perhaps a loss of accuracy as in float-to-int conversions) regardless of which path it takes. The Currency class provides a good example of this. Consider this code: Currency balance = new Currency(50, 35); ulong bal = (ulong) balance; c07.indd 179 30-01-2014 20:13:19 180  ❘  CHAPTER 7  Operators and Casts At present, there is only one way that the compiler can achieve this conversion: by converting the Currency to a float implicitly, then to a ulong explicitly. The float-to-ulong conversion requires an explicit conversion, but that is fine because you have specified one here. Suppose, however, that you then added another cast, to convert implicitly from a Currency to a uint. You will actually do this by modifying the Currency struct by adding the casts both to and from uint. This code is available as the SimpleCurrency2 example: public static implicit operator Currency (uint value) { return new Currency(value, 0); } public static implicit operator uint (Currency value) { return value.Dollars; } Now the compiler has another possible route to convert from Currency to ulong: to convert from Currency to uint implicitly, then to ulong implicitly. Which of these two routes will it take? C# has some precise rules about the best route for the compiler when there are several possibilities. (The rules are not covered in this book, but if you are interested in the details, see the MSDN documentation.) The best answer is that you should design your casts so that all routes give the same answer (other than possible loss of precision), in which case it doesn’t really matter which one the compiler picks. (As it happens in this case, the compiler picks the Currency-to-uint-to-ulong route in preference to Currency-to-float-to-ulong.) To test the SimpleCurrency2 sample, add this code to the test code for SimpleCurrency: try { Currency balance = new Currency(50,35); Console.WriteLine(balance); Console.WriteLine("balance is " + balance); Console.WriteLine("balance is (using ToString()) " + balance.ToString()); uint balance3 = (uint) balance; Console.WriteLine("Converting to uint gives " + balance3); Running the sample now gives you these results: SimpleCurrency2 50 balance is $50.35 balance is (using ToString()) $50.35 Converting to uint gives 50 After converting to float, = 50.35 After converting back to Currency, = $50.34 Now attempt to convert out of range value of -$50.50 to a Currency: Result is $4294967246.00 The output shows that the conversion to uint has been successful, though as expected, you have lost the cents part of the Currency in making this conversion. Casting a negative float to Currency has also produced the expected overflow exception now that the float-to-Currency cast itself defines a checked context. However, the output also demonstrates one last potential problem that you need to be aware of when working with casts. The very first line of output does not display the balance correctly, displaying 50 instead of $50.35. Consider these lines: Console.WriteLine(balance); Console.WriteLine("balance is " + balance); Console.WriteLine("balance is (using ToString()) " + balance.ToString()); c07.indd 180 30-01-2014 20:13:20 Summary  ❘  181 Only the last two lines correctly display the Currency as a string. So what is going on? The problem here is that when you combine casts with method overloads, you get another source of unpredictability. We will look at these lines in reverse order. The third Console.WriteLine() statement explicitly calls the Currency.ToString() method, ensuring that the Currency is displayed as a string. The second does not. However, the string literal "balance is" passed to Console.WriteLine() makes it clear to the compiler that the parameter is to be interpreted as a string. Hence, the Currency.ToString() method is called implicitly. The very first Console.WriteLine() method, however, simply passes a raw Currency struct to Console .WriteLine(). Now, Console.WriteLine() has many overloads, but none of them takes a Currency struct. Therefore, the compiler will start fishing around to see what it can cast the Currency to in order to make it match up with one of the overloads of Console.WriteLine(). As it happens, one of the Console .WriteLine() overloads is designed to display uints quickly and efficiently, and it takes a uint as a parameter — you have now supplied a cast that converts Currency implicitly to uint. In fact, Console.WriteLine() has another overload that takes a double as a parameter and displays the value of that double. If you look closely at the output from the first SimpleCurrency example, you will see that the first line of output displayed Currency as a double, using this overload. In that example, there wasn’t a direct cast from Currency to uint, so the compiler picked Currency-to-float-to-double as its preferred way of matching up the available casts to the available Console.WriteLine() overloads. However, now that there is a direct cast to uint available in SimpleCurrency2, the compiler has opted for that route. The upshot of this is that if you have a method call that takes several overloads and you attempt to pass it a parameter whose data type doesn’t match any of the overloads exactly, then you are forcing the compiler to decide not only what casts to use to perform the data conversion, but also which overload, and hence which data conversion, to pick. The compiler always works logically and according to strict rules, but the results may not be what you expected. If there is any doubt, you are better off specifying which cast to use explicitly. Summary This chapter looked at the standard operators provided by C#, described the mechanics of object equality, and examined how the compiler converts the standard data types from one to another. It also demonstrated how you can implement custom operator support on your data types using operator overloads. Finally, you looked at a special type of operator overload, the cast operator, which enables you to specify how instances of your types are converted to other data types. c07.indd 181 30-01-2014 20:13:20 c07.indd 182 30-01-2014 20:13:20 Delegates, Lambdas, and Events WHAT’s iN THis CHAPTER? ➤➤ Delegates ➤➤ Lambda expressions ➤➤ Closures ➤➤ Events ➤➤ Weak Events WROx.COM CODE DOWNlOADs FOR THis CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ Simple Delegates ➤➤ Bubble Sorter ➤➤ Lambda Expressions ➤➤ Events Sample ➤➤ Weak Events REFERENCiNg METHODs Delegates are the .NET variant of addresses to methods. Compare this to C++, where a function pointer is nothing more than a pointer to a memory location that is not type-safe. You have no idea what a pointer is really pointing to, and items such as parameters and return types are not known. 8 c08.indd 183 30-01-2014 20:13:56 184  ❘  CHAPTER 8  Delegates, Lambdas, and Events This is completely different with .NET; delegates are type-safe classes that define the return types and types of parameters. The delegate class not only contains a reference to a method, but can hold references to multiple methods. Lambda expressions are directly related to delegates. When the parameter is a delegate type, you can use a lambda expression to implement a method that’s referenced from the delegate. This chapter explains the basics of delegates and lambda expressions, and shows you how to implement methods called by delegates with lambda expressions. It also demonstrates how .NET uses delegates as the means of implementing events. Delegates Delegates exist for situations in which you want to pass methods around to other methods. To see what that means, consider this line of code: int i = int.Parse("99"); You are so used to passing data to methods as parameters, as in this example, that you don’t consciously think about it, so the idea of passing methods around instead of data might sound a little strange. However, sometimes you have a method that does something, and rather than operate on data, the method might need to do something that involves invoking another method. To complicate things further, you do not know at compile time what this second method is. That information is available only at runtime and hence will need to be passed in as a parameter to the first method. That might sound confusing, but it should become clearer with a couple of examples: ➤➤ Starting threads and tasks — It is possible in C# to tell the computer to start a new sequence of execution in parallel with what it is currently doing. Such a sequence is known as a thread, and starting one is done using the Start method on an instance of one of the base classes, System .Threading.Thread. If you tell the computer to start a new sequence of execution, you have to tell it where to start that sequence; that is, you have to supply the details of a method in which execution can start. In other words, the constructor of the Thread class takes a parameter that defines the method to be invoked by the thread. ➤➤ Generic library classes — Many libraries contain code to perform various standard tasks. It is usually possible for these libraries to be self-contained, in the sense that you know when you write to the library exactly how the task must be performed. However, sometimes the task contains a subtask, which only the individual client code that uses the library knows how to perform. For example, say that you want to write a class that takes an array of objects and sorts them in ascending order. Part of the sorting process involves repeatedly taking two of the objects in the array and comparing them to see which one should come first. If you want to make the class capable of sorting arrays of any object, there is no way that it can tell in advance how to do this comparison. The client code that hands your class the array of objects must also tell your class how to do this comparison for the particular objects it wants sorted. The client code has to pass your class details of an appropriate method that can be called to do the comparison. ➤➤ Events — The general idea here is that often you have code that needs to be informed when some event takes place. GUI programming is full of situations similar to this. When the event is raised, the runtime needs to know what method should be executed. This is done by passing the method that handles the event as a parameter to a delegate. This is discussed later in this chapter. In C and C++, you can just take the address of a function and pass it as a parameter. There’s no type safety with C. You can pass any function to a method where a function pointer is required. Unfortunately, this direct approach not only causes some problems with type safety, but also neglects the fact that when you are doing object-oriented programming, methods rarely exist in isolation, but usually need to be associated with a class instance before they can be called. Because of these problems, the .NET Framework does not c08.indd 184 30-01-2014 20:13:57 Delegates  ❘  185 syntactically permit this direct approach. Instead, if you want to pass methods around, you have to wrap the details of the method in a new kind of object, a delegate. Delegates, quite simply, are a special type of object — special in the sense that, whereas all the objects defined up to now contain data, a delegate contains the address of a method, or the address of multiple methods. Declaring Delegates When you want to use a class in C#, you do so in two stages. First, you need to define the class — that is, you need to tell the compiler what fields and methods make up the class. Then (unless you are using only static methods), you instantiate an object of that class. With delegates it is the same process. You start by declaring the delegates you want to use. Declaring delegates means telling the compiler what kind of method a delegate of that type will represent. Then, you have to create one or more instances of that delegate. Behind the scenes, the compiler creates a class that represents the delegate. The syntax for declaring delegates looks like this: delegate void IntMethodInvoker(int x); This declares a delegate called IntMethodInvoker, and indicates that each instance of this delegate can hold a reference to a method that takes one int parameter and returns void. The crucial point to understand about delegates is that they are type-safe. When you define the delegate, you have to provide full details about the signature and the return type of the method that it represents. NOTE  One good way to understand delegates is to think of a delegate as something that gives a name to a method signature and the return type. Suppose that you want to define a delegate called TwoLongsOp that will represent a method that takes two longs as its parameters and returns a double. You could do so like this: delegate double TwoLongsOp(long first, long second); Or, to define a delegate that will represent a method that takes no parameters and returns a string, you might write this: delegate string GetAString(); The syntax is similar to that for a method definition, except there is no method body and the definition is prefixed with the keyword delegate. Because what you are doing here is basically defining a new class, you can define a delegate in any of the same places that you would define a class — that is to say, either inside another class, outside of any class, or in a namespace as a top-level object. Depending on how visible you want your definition to be, and the scope of the delegate, you can apply any of the normal access modifiers to delegate definitions — public, private, protected, and so on: NOTE  We really mean what we say when we describe defining a delegate as defining a new class. Delegates are implemented as classes derived from the class System .MulticastDelegate, which is derived from the base class System.Delegate. The C# compiler is aware of this class and uses its delegate syntax to hide the details of the operation of this class. This is another good example of how C# works in conjunction with the base classes to make programming as easy as possible. c08.indd 185 30-01-2014 20:13:57 186  ❘  CHAPTER 8  Delegates, Lambdas, and Events public delegate string GetAString(); After you have defined a delegate, you can create an instance of it so that you can use it to store details about a particular method. NOTE  There is an unfortunate problem with terminology here. When you are talking about classes, there are two distinct terms — class, which indicates the broader definition, and object, which means an instance of the class. Unfortunately, with delegates there is only the one term; delegate can refer to both the class and the object. When you create an instance of a delegate, what you have created is also referred to as a delegate. You need to be aware of the context to know which meaning is being used when we talk about delegates. Using Delegates The following code snippet demonstrates the use of a delegate. It is a rather long-winded way of calling the ToString method on an int (code file GetAStringDemo/Program.cs): private delegate string GetAString(); static void Main() { int x = 40; GetAString firstStringMethod = new GetAString(x.ToString); Console.WriteLine("String is {0}", firstStringMethod()); // With firstStringMethod initialized to x.ToString(), // the above statement is equivalent to saying // Console.WriteLine("String is {0}", x.ToString()); } This code instantiates a delegate of type GetAString and initializes it so it refers to the ToString method of the integer variable x. Delegates in C# always syntactically take a one-parameter constructor, the parameter being the method to which the delegate refers. This method must match the signature with which you originally defined the delegate. In this case, you would get a compilation error if you tried to initialize the variable firstStringMethod with any method that did not take any parameters and return a string. Notice that because int.ToString is an instance method (as opposed to a static one), you need to specify the instance (x) as well as the name of the method to initialize the delegate properly. The next line actually uses the delegate to display the string. In any code, supplying the name of a delegate instance, followed by parentheses containing any parameters, has exactly the same effect as calling the method wrapped by the delegate. Hence, in the preceding code snippet, the Console.WriteLine statement is completely equivalent to the commented-out line. In fact, supplying parentheses to the delegate instance is the same as invoking the Invoke method of the delegate class. Because firstStringMethod is a variable of a delegate type, the C# compiler replaces firstStringMethod with firstStringMethod.Invoke: firstStringMethod(); firstStringMethod.Invoke(); For less typing, at every place where a delegate instance is needed, you can just pass the name of the address. This is known by the term delegate inference. This C# feature works as long as the compiler can resolve c08.indd 186 30-01-2014 20:13:57 Delegates  ❘  187 the delegate instance to a specific type. The example initialized the variable firstStringMethod of type GetAString with a new instance of the delegate GetAString: GetAString firstStringMethod = new GetAString(x.ToString); You can write the same just by passing the method name with the variable x to the variable firstStringMethod: GetAString firstStringMethod = x.ToString; The code that is created by the C# compiler is the same. The compiler detects that a delegate type is required with firstStringMethod, so it creates an instance of the delegate type GetAString and passes the address of the method with the object x to the constructor. NOTE  Be aware that you can’t type the brackets to the method name as x.ToString and pass it to the delegate variable. This would be an invocation of the method. The invocation of x.ToString returns a string object that can’t be assigned to the delegate variable. You can only assign the address of a method to the delegate variable. Delegate inference can be used anywhere a delegate instance is required. Delegate inference can also be used with events because events are based on delegates (as you will see later in this chapter). One feature of delegates is that they are type-safe to the extent that they ensure that the signature of the method being called is correct. However, interestingly, they don’t care what type of object the method is being called against or even whether the method is a static method or an instance method. NOTE  An instance of a given delegate can refer to any instance or static method on any object of any type, provided that the signature of the method matches the signature of the delegate. To demonstrate this, the following example expands the previous code snippet so that it uses the firstStringMethod delegate to call a couple of other methods on another object — an instance method and a static method. For this, you use the Currency struct. The Currency struct has its own overload of ToString and a static method with the same signature to GetCurrencyUnit. This way, the same delegate variable can be used to invoke these methods (code file GetAStringDemo/Currency.cs): struct Currency { public uint Dollars; public ushort Cents; public Currency(uint dollars, ushort cents) { this.Dollars = dollars; this.Cents = cents; } public override string ToString() { return string.Format("${0}.{1,2:00}", Dollars,Cents); } c08.indd 187 30-01-2014 20:13:57 188  ❘  CHAPTER 8  Delegates, Lambdas, and Events public static string GetCurrencyUnit() { return "Dollar"; } public static explicit operator Currency (float value) { checked { uint dollars = (uint)value; ushort cents = (ushort)((value - dollars) * 100); return new Currency(dollars, cents); } } public static implicit operator float (Currency value) { return value.Dollars + (value.Cents / 100.0f); } public static implicit operator Currency (uint value) { return new Currency(value, 0); } public static implicit operator uint (Currency value) { return value.Dollars; } } Now you can use the GetAString instance as follows: private delegate string GetAString(); static void Main() { int x = 40; GetAString firstStringMethod = x.ToString; Console.WriteLine("String is {0}", firstStringMethod()); Currency balance = new Currency(34, 50); // firstStringMethod references an instance method firstStringMethod = balance.ToString; Console.WriteLine("String is {0}", firstStringMethod()); // firstStringMethod references a static method firstStringMethod = new GetAString(Currency.GetCurrencyUnit); Console.WriteLine("String is {0}", firstStringMethod()); } This code shows how you can call a method via a delegate and subsequently reassign the delegate to refer to different methods on different instances of classes, even static methods or methods against instances of different types of class, provided that the signature of each method matches the delegate definition. c08.indd 188 30-01-2014 20:13:57 Delegates  ❘  189 When you run the application, you get the output from the different methods that are referenced by the delegate: String is 40 String is $34.50 String is Dollar However, you still haven’t seen the process of actually passing a delegate to another method. Nor has this actually achieved anything particularly useful yet. It is possible to call the ToString method of int and Currency objects in a much more straightforward way than using delegates. Unfortunately, the nature of delegates requires a fairly complex example before you can really appreciate their usefulness. The next section presents two delegate examples. The first one simply uses delegates to call a couple of different operations. It illustrates how to pass delegates to methods and how you can use arrays of delegates — although arguably it still doesn’t do much that you couldn’t do a lot more simply without delegates. The second, much more complex, example presents a BubbleSorter class, which implements a method to sort arrays of objects into ascending order. This class would be difficult to write without using delegates. Simple Delegate Example This example defines a MathOperations class that uses a couple of static methods to perform two operations on doubles. Then you use delegates to invoke these methods. The math class looks like this: class MathOperations { public static double MultiplyByTwo(double value) { return value * 2; } public static double Square(double value) { return value * value; } } You invokethese methods as follows (code file SimpleDelegate/Program.cs): using System; namespace Wrox.ProCSharp.Delegates { delegate double DoubleOp(double x); class Program { static void Main() { DoubleOp[] operations = { MathOperations.MultiplyByTwo, MathOperations.Square }; for (int i=0; i < operations.Length; i++) { Console.WriteLine("Using operations[{0}]:", i); ProcessAndDisplayNumber(operations[i], 2.0); ProcessAndDisplayNumber(operations[i], 7.94); ProcessAndDisplayNumber(operations[i], 1.414); Console.WriteLine(); c08.indd 189 30-01-2014 20:13:57 190  ❘  CHAPTER 8  Delegates, Lambdas, and Events } } static void ProcessAndDisplayNumber(DoubleOp action, double value) { double result = action(value); Console.WriteLine("Value is {0}, result of operation is {1}", value, result); } } } In this code, you instantiate an array of DoubleOp delegates (remember that after you have defined a delegate class, you can basically instantiate instances just as you can with normal classes, so putting some into an array is no problem). Each element of the array is initialized to refer to a different operation implemented by the MathOperations class. Then, you loop through the array, applying each operation to three different values. This illustrates one way of using delegates — to group methods together into an array so that you can call several methods in a loop. The key lines in this code are the ones in which you actually pass each delegate to the ProcessAndDisplayNumber method, such as here: ProcessAndDisplayNumber(operations[i], 2.0); The preceding passes in the name of a delegate but without any parameters. Given that operations[i] is a delegate, syntactically: ➤➤ operations[i] means the delegate (that is, the method represented by the delegate) ➤➤ operations[i](2.0) means actually call this method, passing in the value in parentheses The ProcessAndDisplayNumber method is defined to take a delegate as its first parameter: static void ProcessAndDisplayNumber(DoubleOp action, double value) Then, when in this method, you call: double result = action(value); This actually causes the method that is wrapped up by the action delegate instance to be called and its return result stored in Result. Running this example gives you the following: SimpleDelegate Using operations[0]: Value is 2, result of operation is 4 Value is 7.94, result of operation is 15.88 Value is 1.414, result of operation is 2.828 Using operations[1]: Value is 2, result of operation is 4 Value is 7.94, result of operation is 63.0436 Value is 1.414, result of operation is 1.999396 Ac tion and Func Delegates Instead of defining a new delegate type with every parameter and return type, you can use the Action and Func delegates. The generic Action delegate is meant to reference a method with void return. This delegate class exists in different variants so that you can pass up to 16 different parameter types. The Action class without the generic parameter is for calling methods without parameters. Action is for calling c08.indd 190 30-01-2014 20:13:57 Delegates  ❘  191 a method with one parameter; Action for a method with two parameters; and Action for a method with eight parameters. The Func delegates can be used in a similar manner. Func allows you to invoke methods with a return type. Similar to Action, Func is defined in different variants to pass up to 16 parameter types and a return type. Func is the delegate type to invoke a method with a return type and without parameters. Func is for a method with one parameter, and Func is for a method with four parameters. The example in the preceding section declared a delegate with a double parameter and a double return type: delegate double DoubleOp(double x); Instead of declaring the custom delegate DoubleOp you can use the Func delegate. You can declare a variable of the delegate type, or as shown here, an array of the delegate type: Func[] operations = { MathOperations.MultiplyByTwo, MathOperations.Square }; and use it with the ProcessAndDisplayNumber() method as a parameter: static void ProcessAndDisplayNumber(Func action, double value) { double result = action(value); Console.WriteLine("Value is {0}, result of operation is {1}", value, result); } BubbleSorter Example You are now ready for an example that shows the real usefulness of delegates. You are going to write a class called BubbleSorter. This class implements a static method, Sort, which takes as its first parameter an array of objects, and rearranges this array into ascending order. For example, if you were to pass it this array of ints, {0, 5, 6, 2, 1}, it would rearrange this array into {0, 1, 2, 5, 6}. The bubble-sorting algorithm is a well-known and very simple way to sort numbers. It is best suited to small sets of numbers, because for larger sets of numbers (more than about 10), far more efficient algorithms are available. It works by repeatedly looping through the array, comparing each pair of numbers and, if necessary, swapping them, so that the largest numbers progressively move to the end of the array. For sorting ints, a method to do a bubble sort might look similar to this: bool swapped = true; do { swapped = false; for (int i = 0; i < sortArray.Length — 1; i++) { if (sortArray[i] > sortArray[i+1])) // problem with this test { int temp = sortArray[i]; sortArray[i] = sortArray[i + 1]; sortArray[i + 1] = temp; swapped = true; } } } while (swapped); c08.indd 191 30-01-2014 20:13:58 192  ❘  CHAPTER 8  Delegates, Lambdas, and Events This is all very well for ints, but you want your Sort method to be able to sort any object. In other words, if some client code hands you an array of Currency structs or any other class or struct that it may have defined, you need to be able to sort the array. This presents a problem with the line if(sortArray[i] < sortArray[i+1]) in the preceding code, because that requires you to compare two objects on the array to determine which one is greater. You can do that for ints, but how do you do it for a new class that doesn’t implement the < operator? The answer is that the client code that knows about the class will have to pass in a delegate wrapping a method that does the comparison. Also, instead of using an int type for the temp variable, a generic Sort method can be implemented using a generic type. With a generic Sort method accepting type T, a comparison method is needed that has two parameters of type T and a return type of bool for the if comparison. This method can be referenced from a Func delegate, where T1 and T2 are the same type: Func. This way, you give your Sort method the following signature: static public void Sort(IList sortArray, Func comparison) The documentation for this method states that comparison must refer to a method that takes two arguments, and returns true if the value of the first argument is smaller than the second one. Now you are all set. Here’s the definition for the BubbleSorter class (code file BubbleSorter/ BubbleSorter.cs): class BubbleSorter { static public void Sort(IList sortArray, Func comparison) { bool swapped = true; do { swapped = false; for (int i = 0; i < sortArray.Count — 1; i++) { if (comparison(sortArray[i+1], sortArray[i])) { T temp = sortArray[i]; sortArray[i] = sortArray[i + 1]; sortArray[i + 1] = temp; swapped = true; } } } while (swapped); } } To use this class, you need to define another class, which you can use to set up an array that needs sorting. For this example, assume that the Mortimer Phones mobile phone company has a list of employees and wants them sorted according to salary. Each employee is represented by an instance of a class, Employee, which looks similar to this (code file BubbleSorter/Employee.cs): class Employee { public Employee(string name, decimal salary) { this.Name = name; this.Salary = salary; } public string Name { get; private set; } public decimal Salary { get; private set; } c08.indd 192 30-01-2014 20:13:58 Delegates  ❘  193 public override string ToString() { return string.Format("{0}, {1:C}", Name, Salary); } public static bool CompareSalary(Employee e1, Employee e2) { return e1.Salary < e2.Salary; } } Note that to match the signature of the Func delegate, you have to define CompareSalary in this class as taking two Employee references and returning a Boolean. In the implementation, the comparison based on salary is performed. Now you are ready to write some client code to request a sort (code file BubbleSorter/Program.cs): using System; namespace Wrox.ProCSharp.Delegates { class Program { static void Main() { Employee[] employees = { new Employee("Bugs Bunny", 20000), new Employee("Elmer Fudd", 10000), new Employee("Daffy Duck", 25000), new Employee("Wile Coyote", 1000000.38m), new Employee("Foghorn Leghorn", 23000), new Employee("RoadRunner", 50000) }; BubbleSorter.Sort(employees, Employee.CompareSalary); foreach (var employee in employees) { Console.WriteLine(employee); } } } } Running this code shows that the Employees are correctly sorted according to salary: BubbleSorter Elmer Fudd, $10,000.00 Bugs Bunny, $20,000.00 Foghorn Leghorn, $23,000.00 Daffy Duck, $25,000.00 RoadRunner, $50,000.00 Wile Coyote, $1,000,000.38 Multicast Delegates So far, each of the delegates you have used wraps just one method call. Calling the delegate amounts to calling that method. If you want to call more than one method, you need to make an explicit call through a delegate more than once. However, it is possible for a delegate to wrap more than one method. Such a c08.indd 193 30-01-2014 20:13:58 194  ❘  CHAPTER 8  Delegates, Lambdas, and Events delegate is known as a multicast delegate. When a multicast delegate is called, it successively calls each method in order. For this to work, the delegate signature should return a void; otherwise, you would only get the result of the last method invoked by the delegate. With a void return type, the Action delegate can be used (code file MulticastDelegates/Program.cs): class Program { static void Main() { Action operations = MathOperations.MultiplyByTwo; operations += MathOperations.Square; In the earlier example, you wanted to store references to two methods, so you instantiated an array of delegates. Here, you simply add both operations into the same multicast delegate. Multicast delegates recognize the operators + and +=. Alternatively, you can expand the last two lines of the preceding code, as in this snippet: Action operation1 = MathOperations.MultiplyByTwo; Action operation2 = MathOperations.Square; Action operations = operation1 + operation2; Multicast delegates also recognize the operators – and -= to remove method calls from the delegate. NOTE  In terms of what’s going on under the hood, a multicast delegate is a class derived from System.MulticastDelegate, which in turn is derived from System .Delegate. System.MulticastDelegate, and has additional members to allow the chaining of method calls into a list. To illustrate the use of multicast delegates, the following code recasts the SimpleDelegate example into a new example, MulticastDelegate. Because you now need the delegate to refer to methods that return void, you have to rewrite the methods in the MathOperations class so they display their results instead of returning them: class MathOperations { public static void MultiplyByTwo(double value) { double result = value * 2; Console.WriteLine("Multiplying by 2: {0} gives {1}", value, result); } public static void Square(double value) { double result = value * value; Console.WriteLine("Squaring: {0} gives {1}", value, result); } } To accommodate this change, you also have to rewrite ProcessAndDisplayNumber: static void ProcessAndDisplayNumber(Action action, double value) { Console.WriteLine(); c08.indd 194 30-01-2014 20:13:58 Delegates  ❘  195 Console.WriteLine("ProcessAndDisplayNumber called with value = {0}", value); action(value); } Now you can try out your multicast delegate: static void Main() { Action operations = MathOperations.MultiplyByTwo; operations += MathOperations.Square; ProcessAndDisplayNumber(operations, 2.0); ProcessAndDisplayNumber(operations, 7.94); ProcessAndDisplayNumber(operations, 1.414); Console.WriteLine(); } Each time ProcessAndDisplayNumber is called now, it will display a message saying that it has been called. Then the following statement will cause each of the method calls in the action delegate instance to be called in succession: action(value); Running the preceding code produces this result: MulticastDelegate ProcessAndDisplayNumber called with value = 2 Multiplying by 2: 2 gives 4 Squaring: 2 gives 4 ProcessAndDisplayNumber called with value = 7.94 Multiplying by 2: 7.94 gives 15.88 Squaring: 7.94 gives 63.0436 ProcessAndDisplayNumber called with value = 1.414 Multiplying by 2: 1.414 gives 2.828 Squaring: 1.414 gives 1.999396 If you are using multicast delegates, be aware that the order in which methods chained to the same delegate will be called is formally undefined. Therefore, avoid writing code that relies on such methods being called in any particular order. Invoking multiple methods by one delegate might cause an even bigger problem. The multicast delegate contains a collection of delegates to invoke one after the other. If one of the methods invoked by a delegate throws an exception, the complete iteration stops. Consider the following MulticastIteration example. Here, the simple delegate Action that returns void without arguments is used. This delegate is meant to invoke the methods One and Two, which fulfill the parameter and return type requirements of the delegate. Be aware that method One throws an exception (code file MulticastDelegateWithIteration/Program.cs): using System; namespace Wrox.ProCSharp.Delegates { class Program { static void One() { c08.indd 195 30-01-2014 20:13:58 196  ❘  CHAPTER 8  Delegates, Lambdas, and Events Console.WriteLine("One"); throw new Exception("Error in one"); } static void Two() { Console.WriteLine("Two"); } In the Main method, delegate d1 is created to reference method One; next, the address of method Two is added to the same delegate. d1 is invoked to call both methods. The exception is caught in a try/catch block: static void Main() { Action d1 = One; d1 += Two; try { d1(); } catch (Exception) { Console.WriteLine("Exception caught"); } } } } Only the first method is invoked by the delegate. Because the first method throws an exception, iterating the delegates stops here and method Two() is never invoked. The result might differ because the order of calling the methods is not defined: One Exception Caught NOTE  Errors and exceptions are explained in detail in Chapter 16, “Errors and Exceptions.” In such a scenario, you can avoid the problem by iterating the list on your own. The Delegate class defines the method GetInvocationList that returns an array of Delegate objects. You can now use this delegate to invoke the methods associated with them directly, catch exceptions, and continue with the next iteration: static void Main() { Action d1 = One; d1 += Two; Delegate[] delegates = d1.GetInvocationList(); foreach (Action d in delegates) { try { d(); c08.indd 196 30-01-2014 20:13:58 Delegates  ❘  197 } catch (Exception) { Console.WriteLine("Exception caught"); } } } When you run the application with the code changes, you can see that the iteration continues with the next method after the exception is caught: One Exception caught Two Anonymous Methods Up to this point, a method must already exist for the delegate to work (that is, the delegate is defined with the same signature as the method(s) it will be used with). However, there is another way to use delegates — with anonymous methods. An anonymous method is a block of code that is used as the parameter for the delegate. The syntax for defining a delegate with an anonymous method doesn’t change. It’s when the delegate is instantiated that things change. The following very simple console application shows how using an anonymous method can work (code file AnonymousMethods/Program.cs): using System; namespace Wrox.ProCSharp.Delegates { class Program { static void Main() { string mid = ", middle part,"; Func anonDel = delegate(string param) { param += mid; param += " and this was added to the string."; return param; }; Console.WriteLine(anonDel("Start of string")); } } } The delegate Func takes a single string parameter and returns a string. anonDel is a variable of this delegate type. Instead of assigning the name of a method to this variable, a simple block of code is used, prefixed by the delegate keyword, followed by a string parameter. As you can see, the block of code uses a method-level string variable, mid, which is defined outside of the anonymous method and adds it to the parameter that was passed in. The code then returns the string value. When the delegate is called, a string is passed in as the parameter and the returned string is output to the console. The benefit of using anonymous methods is that it reduces the amount of code you have to write. You don’t need to define a method just to use it with a delegate. This becomes evident when defining the delegate for c08.indd 197 30-01-2014 20:13:58 198  ❘  CHAPTER 8  Delegates, Lambdas, and Events an event (events are discussed later in this chapter), and it helps reduce the complexity of code, especially where several events are defined. With anonymous methods, the code does not perform faster. The compiler still defines a method; the method just has an automatically assigned name that you don’t need to know. A couple of rules must be followed when using anonymous methods. You can’t have a jump statement (break, goto, or continue) in an anonymous method that has a target outside of the anonymous method. The reverse is also true — a jump statement outside the anonymous method cannot have a target inside the anonymous method. Unsafe code cannot be accessed inside an anonymous method, and the ref and out parameters that are used outside of the anonymous method cannot be accessed. Other variables defined outside of the anonymous method can be used. If you have to write the same functionality more than once, don’t use anonymous methods. In this case, instead of duplicating the code, write a named method. You only have to write it once and reference it by its name. Beginning with C# 3.0, you can use lambda expressions instead of writing anonymous methods. Lambda Expressions Since C# 3.0, you can use a different syntax for assigning code implementation to delegates: lambda expressions. Lambda expressions can be used whenever you have a delegate parameter type. The previous example using anonymous methods is modified here to use a lambda expression. NOTE  The syntax of lambda expressions is simpler than the syntax of anonymous methods. In a case where a method to be invoked has parameters and you don’t need the parameters, the syntax of anonymous methods is simpler, as you don’t need to supply parameters in that case. using System; namespace Wrox.ProCSharp.Delegates { class Program { static void Main() { string mid = ", middle part,"; Func lambda = param => { param += mid; param += " and this was added to the string."; return param; }; Console.WriteLine(lambda("Start of string")); } } } The left side of the lambda operator, =>, lists the parameters needed. The right side following the lambda operator defines the implementation of the method assigned to the variable lambda. c08.indd 198 30-01-2014 20:13:59 Lambda Expressions  ❘  199 Parameters With lambda expressions there are several ways to define parameters. If there’s only one parameter, just the name of the parameter is enough. The following lambda expression uses the parameter named s. Because the delegate type defines a string parameter, s is of type string. The implementation invokes the String .Format method to return a string that is finally written to the console when the delegate is invoked: change uppercase TEST: Func oneParam = s => String.Format("change uppercase {0}", s.ToUpper()); Console.WriteLine(oneParam("test")); If a delegate uses more than one parameter, you can combine the parameter names inside brackets. Here, the parameters x and y are of type double as defined by the Func delegate: Func twoParams = (x, y) => x * y; Console.WriteLine(twoParams(3, 2)); For convenience, you can add the parameter types to the variable names inside the brackets. If the compiler can’t match an overloaded version, using parameter types can help resolve the matching delegate: Func twoParamsWithTypes = (double x, double y) => x * y; Console.WriteLine(twoParamsWithTypes(4, 2)); Multiple Code Lines If the lambda expression consists of a single statement, a method block with curly brackets and a return statement are not needed. There’s an implicit return added by the compiler: Func square = x => x * x; It’s completely legal to add curly brackets, a return statement, and semicolons. Usually it’s just easier to read without them: Func square = x => { return x * x; } However, if you need multiple statements in the implementation of the lambda expression, curly brackets and the return statement are required: Func lambda = param => { param += mid; param += " and this was added to the string."; return param; }; Closures With lambda expressions you can access variables outside the block of the lambda expression. This is known by the term closure. Closures are a great feature but they can also be very dangerous if not used correctly. c08.indd 199 30-01-2014 20:13:59 200  ❘  CHAPTER 8  Delegates, Lambdas, and Events In the following example here, a lambda expression of type Func requires one int parameter and returns an int. The parameter for the lambda expression is defined with the variable x. The implementation also accesses the variable someVal, which is outside the lambda expression. As long as you do not assume that the lambda expression creates a new method that is used later when f is invoked, this might not look confusing at all. Looking at this code block, the returned value calling f should be the value from x plus 5, but this might not be the case: int someVal = 5; Func f = x => x + someVal; Assuming the variable someVal is later changed, and then the lambda expression invoked, the new value of someVal is used. The result here invoking f(3) is 10: someVal = 7; Console.WriteLine(f(3)); In particular, when the lambda expression is invoked by a separate thread, you might not know when the invocation happened and thus what value the outside variable currently has. Now you might wonder how it is possible at all to access variables outside of the lambda expression from within the lambda expression. To understand this, consider what the compiler does when you define a lambda expression. With the lambda expression x => x + someVal, the compiler creates an anonymous class that has a constructor to pass the outer variable. The constructor depends on how many variables you access from the outside. With this simple example, the constructor accepts an int. The anonymous class contains an anonymous method that has the implementation as defined by the lambda expression, with the parameters and return type: public class AnonymousClass { private int someVal; public AnonymousClass(int someVal) { this.someVal = someVal; } public int AnonymousMethod(int x) { return x + someVal; } } Using the lambda expression and invoking the method creates an instance of the anonymous class and passes the value of the variable from the time when the call is made. Closures with Foreach Statements foreach statements have an important change with C# 5 in regard to closures. In the following example, first a list named values is filled with the values 10, 20, and 30. The funcs variable references a generic list in which each object references a delegate of type Func. The elements of the funcs list are added within the first foreach statement. The function added to the items is defined with a lambda expression. This lambda expression makes use of the variable val that is declared outside of the lambda as a loop variable with the foreach statement. The second foreach statement iterates through the list of funcs to invoke every method that is referenced: var values = new List() { 10, 20, 30 }; var funcs = new List>(); foreach (var val in values) c08.indd 200 30-01-2014 20:13:59 Events  ❘  201 { funcs.Add(() => val); } foreach (var f in funcs) { Console.WriteLine((f())); } The outcome of this code snippet changed with C# 5. Using C# 4 or earlier versions of the compiler, 30 is written to the console three times. Using a closure with the first foreach loop, the functions that are created don’t take the value of the val variable during the time of the iteration, but instead when the functions are invoked. As you’ve already seen in Chapter 6, “Arrays and Tuples,” the compiler creates a while loop out from the foreach statement. With C# 4 the compiler defines the loop variable outside of the while loop and reuses it with every iteration. Thus, at the end of the loop the variable has the value from the last iteration. To get 10, 20, 30 with the result of the code using C# 4, it’s necessary to change the code to use a local variable that is passed to the lambda expression. Here, a different value is retained with every iteration. var values = new List() { 10, 20, 30 }; var funcs = new List>(); foreach (var val in values) { var v = val; funcs.Add(() => v); } foreach (var f in funcs) { Console.WriteLine((f())); } Using C# 5 the code change to have a local variable is no longer necessary. C# now creates the loop variable differently locally within the block of the while loop and thus the value is retained automatically. You just need to be aware of these different behaviors of C# 4 and 5. NOTE  Lambda expressions can be used anywhere the type is a delegate. Another use of lambda expressions is when the type is Expression or Expression. , in which case the compiler creates an expression tree. This feature is discussed in Chapter 11, “Language Integrated Query.” Events Events are based on delegates and offer a publish/subscribe mechanism to delegates. You can find events everywhere across the framework. In Windows applications, the Button class offers the Click event. This type of event is a delegate. A handler method that is invoked when the Click event is fired needs to be defined, with the parameters as defined by the delegate type. In the code example shown in this section, events are used to connect CarDealer and Consumer classes. The CarDealer offers an event when a new car arrives. The Consumer class subscribes to the event to be informed when a new car arrives. Event Publisher We start with a CarDealer class that offers a subscription based on events. CarDealer defines the event named NewCarInfo of type EventHandler with the event keyword. Inside the method NewCar, the event NewCarInfo is fired by invoking the method RaiseNewCarInfo. c08.indd 201 30-01-2014 20:13:59 202  ❘  CHAPTER 8  Delegates, Lambdas, and Events The implementation of this method verifies if the delegate is not null, and raises the event (code file EventSample/CarDealer.cs): using System; namespace Wrox.ProCSharp.Delegates { public class CarInfoEventArgs: EventArgs { public CarInfoEventArgs(string car) { this.Car = car; } public string Car { get; private set; } } public class CarDealer { public event EventHandler NewCarInfo; public void NewCar(string car) { Console.WriteLine("CarDealer, new car {0}", car); RaiseNewCarInfo(car); } protected virtual void RaiseNewCarInfo(string car) { EventHandler newCarInfo = NewCarInfo; if (newCarInfo != null) { newCarInfo(this, new CarInfoEventArgs(car)); } } } } The class CarDealer offers the event NewCarInfo of type EventHandler. As a convention, events typically use methods with two parameters; the first parameter is an object and contains the sender of the event, and the second parameter provides information about the event. The second parameter is different for various event types. .NET 1.0 defined several hundred delegates for events for all different data types. That’s no longer necessary with the generic delegate EventHandler. EventHandler defines a handler that returns void and accepts two parameters. With EventHandler, the first parameter needs to be of type object, and the second parameter is of type T. EventHandler also defines a constraint on T; it must derive from the base class EventArgs, which is the case with CarInfoEventArgs: public event EventHandler NewCarInfo; The delegate EventHandler is defined as follows: public delegate void EventHandler(object sender, TEventArgs e) where TEventArgs: EventArgs Defining the event in one line is a C# shorthand notation. The compiler creates a variable of the delegate type EventHandler and adds methods to subscribe and unsubscribe from the delegate. The long form of the shorthand notation is shown next. This is very similar to auto-properties c08.indd 202 30-01-2014 20:13:59 Events  ❘  203 and full properties. With events, the add and remove keywords are used to add and remove a handler to the delegate: private EventHandler newCarInfo; public event EventHandler NewCarInfo { add { newCarInfo += value; } remove { newCarInfo -= value; } } NOTE  The long notation to define events is useful if more needs to be done than just adding and removing the event handler, such as adding synchronization for multiple thread access. The WPF controls make use of the long notation to add bubbling and tunneling functionality with the events. You can read more about event bubbling and tunneling events in Chapter 29, “Core XAML.” The class CarDealer fires the event in the method RaiseNewCarInfo. Using the delegate NewCarInfo with brackets invokes all the handlers that are subscribed to the event. Remember, as shown with multicast delegates, the order of the methods invoked is not guaranteed. To have more control over calling the handler methods you can use the Delegate class method GetInvocationList to access every item in the delegate list and invoke each on its own, as shown earlier. Before firing the event, it is necessary to check whether the delegate NewCarInfo is not null. If no one subscribed, the delegate is null: protected virtual void RaiseNewCarInfo(string car) { var newCarInfo = NewCarInfo; if (newCarInfo != null) { newCarInfo(this, new CarInfoEventArgs(car)); } } Event Listener The class Consumer is used as the event listener. This class subscribes to the event of the CarDealer and defines the method NewCarIsHere that in turn fulfills the requirements of the EventHandler delegate with parameters of type object and CarInfoEventArgs (code file EventsSample/Consumer.cs): using System; namespace Wrox.ProCSharp.Delegates { public class Consumer { c08.indd 203 30-01-2014 20:13:59 204  ❘  CHAPTER 8  Delegates, Lambdas, and Events private string name; public Consumer(string name) { this.name = name; } public void NewCarIsHere(object sender, CarInfoEventArgs e) { Console.WriteLine("{0}: car {1} is new", name, e.Car); } } } Now the event publisher and subscriber need to connect. This is done by using the NewCarInfo event of the CarDealer to create a subscription with +=. The consumer michael subscribes to the event, then the consumer sebastian, and next michael unsubscribes with -= (code file EventsSample/Program.cs): namespace Wrox.ProCSharp.Delegates { class Program { static void Main() { var dealer = new CarDealer(); var michael = new Consumer("Michael"); dealer.NewCarInfo += michael.NewCarIsHere; dealer.NewCar("Ferrari"); var sebastian = new Consumer("Sebastian"); dealer.NewCarInfo += sebastian.NewCarIsHere; dealer.NewCar("Mercedes"); dealer.NewCarInfo -= michael.NewCarIsHere; dealer.NewCar("Red Bull Racing"); } } } Running the application, a Ferrari arrived and Michael was informed. Because after that Sebastian registers for the subscription as well, both Michael and Sebastian are informed about the new Mercedes. Then Michael unsubscribes and only Sebastian is informed about the Red Bull: CarDealer, new car Ferrari Michael: car Ferrari is new CarDealer, new car Mercedes Michael: car Mercedes is new Sebastian: car Mercedes is new CarDealer, new car Red Bull Sebastian: car Red Bull is new Weak Events With events, the publisher and listener are directly connected. This can be a problem with garbage collection. For example, if a listener is not directly referenced any more, there’s still a reference from the publisher. The garbage collector cannot clean up memory from the listener, as the publisher still holds a reference and fires events to the listener. c08.indd 204 30-01-2014 20:14:00 Events  ❘  205 This strong connection can be resolved by using the weak event pattern and using the WeakEventManager as an intermediary between the publisher and listeners. The preceding example with the CarDealer as publisher and the Consumer as listener is modified in this section to use the weak event pattern. Weak Event Manager To use weak events you need to create a class that derives from WeakEventManager, which is defined in the namespace System.Windows in the assembly WindowsBase. The class WeakCarInfoEventManager is the weak event manager class that manages the connection between the publisher and the listener for the NewCarInfo event. This class implements a singleton pattern so that only one instance is created. The static property CurrentManager creates an object of type WeakCarInfoEventManager if it doesn’t exist, and returns a reference to it. WeakCarInfoEventManager .CurrentManager is used to access the singleton object from the WeakCarInfoEventManager. With the weak event pattern, the weak event manager class needs the static methods AddListener and RemoveListener. The listener is connected and disconnected to the events of the publisher with these methods, instead of using the events from the publisher directly. The listener also needs to implement the interface IWeakEventListener, which is shown shortly. With the AddListener and RemoveListener methods, methods from the base class WeakEventManager are invoked to add and remove the listeners. With the WeakCarInfoEventManager class you also need to override the StartListening and StopListening methods from the base class. StartListening is called when the first listener is added, StopListening when the last listener is removed. StartListening and StopListening subscribes and unsubscribes, respectively, a method from the weak event manager to listen for the event from the publisher. In case the weak event manager class needs to connect to different publisher types, you can check the type information from the source object before doing the cast. The event is then forwarded to the listeners by calling the DeliverEvent method from the base class, which in turn invokes the method ReceiveWeakEvent from the IWeakEventListener interface in the listeners (code file WeakEventsSample/ WeakCarInfoEventManger.cs): using System.Windows; namespace Wrox.ProCSharp.Delegates { public class WeakCarInfoEventManager: WeakEventManager { public static void AddListener(object source, IWeakEventListener listener) { CurrentManager.ProtectedAddListener(source, listener); } public static void RemoveListener(object source, IWeakEventListener listener) { CurrentManager.ProtectedRemoveListener(source, listener); } NOTE  With subscribers that are created dynamically, in order to not be in danger of having resource leaks, you need to pay special attention to events. That is, you need to either ensure that you unsubscribe events before the subscribers go out of scope (are not needed any longer), or use weak events. c08.indd 205 30-01-2014 20:14:00 206  ❘  CHAPTER 8  Delegates, Lambdas, and Events public static WeakCarInfoEventManager CurrentManager { get { var manager = GetCurrentManager(typeof(WeakCarInfoEventManager)) as WeakCarInfoEventManager; if (manager == null) { manager = new WeakCarInfoEventManager(); SetCurrentManager(typeof(WeakCarInfoEventManager), manager); } return manager; } } protected override void StartListening(object source) { (source as CarDealer).NewCarInfo += CarDealer_NewCarInfo; } void CarDealer_NewCarInfo(object sender, CarInfoEventArgs e) { DeliverEvent(sender, e); } protected override void StopListening(object source) { (source as CarDealer).NewCarInfo = CarDealer_NewCarInfo; } } } NOTE  WPF makes use of the weak event pattern with the event manager classes: CollectionChangedEventManager, CurrentChangedEventManager, CurrentChangingEventManager, PropertyChangedEventManager, DataChangedEventManager, and LostFocusEventManager. With the publisher class CarDealer there’s no need to change anything. It has the same implementation as before. Event Listener The listener needs to be changed to implement the interface IWeakEventListener. This interface defines the method ReceiveWeakEvent that is called from the weak event manager when the event arrives. The method implementation acts as a proxy and in turn invokes the method NewCarIsHere (code file WeakEventsSample/Consumer.cs): using System; using System.Windows; namespace Wrox.ProCSharp.Delegates { public class Consumer: IWeakEventListener { private string name; public Consumer(string name) c08.indd 206 30-01-2014 20:14:00 Events  ❘  207 { this.name = name; } public void NewCarIsHere(object sender, CarInfoEventArgs e) { Console.WriteLine("{0}: car {1} is new", name, e.Car); } bool IWeakEventListener.ReceiveWeakEvent(Type managerType, object sender, EventArgs e) { NewCarIsHere(sender, e as CarInfoEventArgs); return true; } } } Inside the Main method, where the publisher and listeners are connected, the connection is now made by using the static AddListener and RemoveListener methods from the WeakCarInfoEventManager class (code file WeakEventsSample/Program.cs): static void Main() { var dealer = new CarDealer(); var michael = new Consumer("Michael"); WeakCarInfoEventManager.AddListener(dealer, michael); dealer.NewCar("Mercedes"); var sebastian = new Consumer("Sebastian"); WeakCarInfoEventManager.AddListener(dealer, sebastian); dealer.NewCar("Ferrari"); WeakCarInfoEventManager.RemoveListener(dealer, michael); dealer.NewCar("Red Bull Racing"); } With this additional work of implementing the weak event pattern, the publisher and listeners are no longer strongly connected. When a listener is not referenced anymore, it can be garbage collected. Generic Weak Event Manager .NET 4.5 has a new implementation of a weak event manager. The generic class WeakEventManager derives from the base class WeakEventManager and makes dealing with weak events a lot easier. Using this class it’s no longer necessary to implement a custom weak event manager class for every event, nor is it necessary that the consumer implements the interface IWeakEventsListener. All that is required is using the generic weak event manager on subscribing to the events. The main program to subscribe to the events is now changed to use the generic WeakEventManager with the event source being the CarDealer type, and the event args that are passed with the event the CarInfoEventArgs type. The class defines the AddHandler method to subscribe to an event, and the RemoveHandler method to unsubscribe. Then the program works as before but with a lot less code: var dealer = new CarDealer(); var michael = new Consumer("Michael"); c08.indd 207 30-01-2014 20:14:00 208  ❘  CHAPTER 8  Delegates, Lambdas, and Events WeakEventManager.AddHandler(dealer, “NewCarInfo”, michael.NewCarIsHere); dealer.NewCar("Mercedes"); var sebastian = new Consumer("Sebastian"); WeakEventManager.AddHandler(dealer, “NewCarInfo”, sebastian.NewCarIsHere); dealer.NewCar("Ferrari"); WeakEventManager.RemoveHandler(dealer, “NewCarInfo”, michael.NewCarIsHere); dealer.NewCar("Red Bull Racing"); Summary This chapter provided the basics of delegates, lambda expressions, and events. You learned how to declare a delegate and add methods to the delegate list; you learned how to implement methods called by delegates with lambda expressions; and you learned the process of declaring event handlers to respond to an event, as well as how to create a custom event and use the patterns for raising the event. As a .NET developer, you will use delegates and events extensively, especially when developing Windows applications. Events are the means by which the .NET developer can monitor the various Windows messages that occur while the application is executing. Otherwise, you would have to monitor the WndProc function and catch the WM_MOUSEDOWN message instead of getting the mouse Click event for a button. Using delegates and events in the design of a large application can reduce dependencies and the coupling of layers. This enables you to develop components that have a higher reusability factor. Lambda expressions are C# language features on delegates. With these, you can reduce the amount of code you need to write. Lambda expressions are not only used with delegates, as you will see in Chapter 11, “Language Integrated Query.” The next chapter covers the use of strings and regular expressions. c08.indd 208 30-01-2014 20:14:00 Strings and Regular Expressions WHAT’S in THiS CHAPTER? ➤➤ Building strings ➤➤ Formatting expressions ➤➤ Using regular expressions WRox.Com CodE doWnloAdS FoR THiS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ Encoder.cs ➤➤ Encoder2.cs ➤➤ FormattableVector.cs ➤➤ RegularExpressionPlayground.cs ➤➤ StringEncoder.cs Strings have been used consistently since the beginning of this book, but you might not have realized that the stated mapping that the string keyword in C# actually refers to is the System.String .NET base class. System.String is a very powerful and versatile class, but it is by no means the only string- related class in the .NET armory. This chapter begins by reviewing the features of System.String and then looks at some nifty things you can do with strings using some of the other .NET classes — in particular those in the System.Text and System.Text.RegularExpressions namespaces. This chapter covers the following areas: ➤➤ Building strings — If you’re performing repeated modifi cations on a string — for example, to build a lengthy string prior to displaying it or passing it to some other method or application — the String class can be very ineffi cient. When you fi nd yourself in this kind of situation, another class, System.Text.StringBuilder, is more suitable because it has been designed exactly for this scenario. ➤➤ Formatting expressions — This chapter takes a closer look at the formatting expressions that have been used in the Console.WriteLine() method throughout the past few chapters. These formatting expressions are processed using two useful interfaces, IFormatProvider 9 c09.indd 209 30-01-2014 20:14:31 210  ❘  CHAPTER 9  Strings and Regular Expressions and IFormattable. By implementing these interfaces on your own classes, you can define your own formatting sequences so that Console.WriteLine() and similar classes display the values of your classes in whatever way you specify. ➤➤ Regular expressions — .NET also offers some very sophisticated classes that deal with cases in which you need to identify or extract substrings that satisfy certain fairly sophisticated criteria; for example, finding all occurrences within a string where a character or set of characters is repeated: finding all words that begin with “s” and contain at least one “n:” or strings that adhere to an employee ID or a social security number construction. Although you can write methods to perform this kind of processing using the String class, writing such methods is cumbersome. Instead, some classes, specifically those from System.Text.RegularExpressions, are designed to perform this kind of processing. Examining System.String Before digging into the other string classes, this section briefly reviews some of the available methods in the String class itself. System.String is a class specifically designed to store a string and allow a large number of operations on the string. In addition, due to the importance of this data type, C# has its own keyword and associated syntax to make it particularly easy to manipulate strings using this class. You can concatenate strings using operator overloads: string message1 = "Hello"; // returns "Hello" message1 += ", There"; // returns "Hello, There" string message2 = message1 + "!"; // returns "Hello, There!" C# also allows extraction of a particular character using an indexer-like syntax: string message = "Hello"; char char4 = message[4]; // returns 'o'. Note the string is zero-indexed This enables you to perform such common tasks as replacing characters, removing whitespace, and changing case. The following table introduces the key methods. Method Description Compare Compares the contents of strings, taking into account the culture (locale) in assessing equivalence between certain characters. CompareOrdinal Same as Compare but doesn’t take culture into account. Concat Combines separate string instances into a single instance. CopyTo Copies a specific number of characters from the selected index to an entirely new instance of an array. Format Formats a string containing various values and specifiers for how each value should be formatted. IndexOf Locates the first occurrence of a given substring or character in the string. IndexOfAny Locates the first occurrence of any one of a set of characters in a string. Insert Inserts a string instance into another string instance at a specified index. Join Builds a new string by combining an array of strings. LastIndexOf Same as IndexOf but finds the last occurrence. LastIndexOfAny Same as IndexOf Any but finds the last occurrence. c09.indd 210 30-01-2014 20:14:31 Examining System.String  ❘  211 PadLeft Pads out the string by adding a specified repeated character to the left side of the string. PadRight Pads out the string by adding a specified repeated character to the right side of the string. Replace Replaces occurrences of a given character or substring in the string with another character or substring. Split Splits the string into an array of substrings; the breaks occur wherever a given character occurs. Substring Retrieves the substring starting at a specified position in a string. ToLower Converts the string to lowercase. ToUpper Converts the string to uppercase. Trim Removes leading and trailing whitespace. Note  Please note that this table is not comprehensive; it is intended to give you an idea of the features offered by strings. Building Strings As you have seen, String is an extremely powerful class that implements a large number of very useful methods. However, the String class has a shortcoming that makes it very inefficient for making repeated modifications to a given string — it is actually an immutable data type, which means that after you initialize a string object, that string object can never change. The methods and operators that appear to modify the contents of a string actually create new strings, copying across the contents of the old string if necessary. For example, consider the following code: string greetingText = "Hello from all the guys at Wrox Press. "; greetingText += "We do hope you enjoy this book as much as we enjoyed writing it."; When this code executes, first an object of type System.String is created and initialized to hold the text Hello from all the guys at Wrox Press. (Note the space after the period.) When this happens, the .NET runtime allocates just enough memory in the string to hold this text (39 chars), and the variable greetingText is set to refer to this string instance. In the next line, syntactically it looks like more text is being added onto the string, but it is not. Instead, a new string instance is created with just enough memory allocated to store the combined text — that’s 103 characters in total. The original text, Hello from all the people at Wrox Press., is copied into this new string instance along with the extra text: We do hope you enjoy this book as much as we enjoyed writing it. Then, the address stored in the variable greetingText is updated, so the variable correctly points to the new String object. The old String object is now unreferenced — there are no variables that refer to it — so it will be removed the next time the garbage collector comes along to clean out any unused objects in your application. By itself, that doesn’t look too bad, but suppose you wanted to create a very simple encryption scheme by adding 1 to the ASCII value of each character in the string. This would change the string to Ifmmp gspn bmm uif hvst bu Xspy Qsftt. Xf ep ipqf zpv fokpz uijt cppl bt nvdi bt xf fokpzfe xsjujoh ju. Several ways of doing this exist, but the simplest and (if you are restricting yourself to using the String class) almost certainly the most efficient way is to use the String.Replace() method, which c09.indd 211 30-01-2014 20:14:31 212  ❘  CHAPTER 9  Strings and Regular Expressions replaces all occurrences of a given substring in a string with another substring. Using Replace(), the code to encode the text looks like this: string greetingText = "Hello from all the guys at Wrox Press. "; greetingText += "We do hope you enjoy this book as much as we enjoyed writing it."; for(int i = 'z'; i>= 'a'; i--) { char old1 = (char)i; char new1 = (char)(i+1); greetingText = greetingText.Replace(old1, new1); } for(int i = 'Z'; i>='A'; i--) { char old1 = (char)i; char new1 = (char)(i+1); greetingText = greetingText.Replace(old1, new1); } Console.WriteLine("Encoded:\n" + greetingText); Note  Simply this code does not wrap Z to A or z to a. These letters are encoded to [ and {, respectively. In this example, the Replace() method works in a fairly intelligent way, to the extent that it won’t actually create a new string unless it actually makes changes to the old string. The original string contained 23 different lowercase characters and three different uppercase ones. The Replace() method will therefore have allocated a new string 26 times in total, with each new string storing 103 characters. That means because of the encryption process, there will be string objects capable of storing a combined total of 2,678 characters now sitting on the heap waiting to be garbagecollected! Clearly, if you use strings to do text processing extensively, your applications will run into severe performance problems. To address this kind of issue, Microsoft supplies the System.Text.StringBuilder class. StringBuilder is not as powerful as String in terms of the number of methods it supports. The processing you can do on a StringBuilder is limited to substitutions and appending or removing text from strings. However, it works in a much more efficient way. When you construct a string using the String class, just enough memory is allocated to hold the string object. The StringBuilder, however, normally allocates more memory than is actually needed. You, as a developer, have the option to indicate how much memory the StringBuilder should allocate; but if you do not, the amount defaults to a value that varies according to the size of the string with which the StringBuilder instance is initialized. The StringBuilder class has two main properties: ➤➤ Length — Indicates the length of the string that it actually contains ➤➤ Capacity — Indicates the maximum length of the string in the memory allocation Any modifications to the string take place within the block of memory assigned to the StringBuilder instance, which makes appending substrings and replacing individual characters within strings very efficient. Removing or inserting substrings is inevitably still inefficient because it means that the following c09.indd 212 30-01-2014 20:14:32 Examining System.String  ❘  213 part of the string has to be moved. Only if you perform an operation that exceeds the capacity of the string is it necessary to allocate new memory and possibly move the entire contained string. In adding extra capacity, based on our experiments the StringBuilder appears to double its capacity if it detects that the capacity has been exceeded and no new value for capacity has been set. For example, if you use a StringBuilder object to construct the original greeting string, you might write this code: StringBuilder greetingBuilder = new StringBuilder("Hello from all the guys at Wrox Press. ", 150); greetingBuilder.AppendFormat("We do hope you enjoy this book as much as we enjoyed writing it"); note  To use the StringBuilder class, you need a Syste m.Te xt reference in your code. This code sets an initial capacity of 150 for the StringBuilder. It is always a good idea to set a capacity that covers the likely maximum length of a string, to ensure that the StringBuilder does not need to relocate because its capacity was exceeded. By default, the capacity is set to 16. Theoretically, you can set a number as large as the number you pass in an int, although the system will probably complain that it does not have enough memory if you actually try to allocate the maximum of two billion characters (the theoretical maximum that a StringBuilder instance is allowed to contain). When the preceding code is executed, it first creates a StringBuilder object that looks like Figure 9-1. Console.WriteLine("The double is {0, 10:E} and the int contains {1}", d, i) String.Format("The double is {0, 10:E} and the int contains {1}", d, i) StringBuilder ("The double is") StringBuilder.AppendFormat ("{0, 10:E}", d) StringBuilder.Append (" and the int contains ") StringBuilder.AppendFormat ("{1}”, i) Figure 9-1 c09.indd 213 30-01-2014 20:14:33 214  ❘  CHAPTER 9  Strings and Regular Expressions Then, on calling the AppendFormat() method, the remaining text is placed in the empty space, without the need to allocate more memory. However, the real efficiency gain from using a StringBuilder is realized when you make repeated text substitutions. For example, if you try to encrypt the text in the same way as before, you can perform the entire encryption without allocating any more memory whatsoever: StringBuilder greetingBuilder = new StringBuilder("Hello from all the guys at Wrox Press. ", 150); greetingBuilder.AppendFormat("We do hope you enjoy this book as much as we " + "enjoyed writing it"); Console.WriteLine("Not Encoded:\n" + greetingBuilder); for(int i = 'z'; i>='a'; i--) { char old1 = (char)i; char new1 = (char)(i+1); greetingBuilder = greetingBuilder.Replace(old1, new1); } for(int i = 'Z'; i>='A'; i--) { char old1 = (char)i; char new1 = (char)(i+1); greetingBuilder = greetingBuilder.Replace(old1, new1); } Console.WriteLine("Encoded:\n" + greetingBuilder); This code uses the StringBuilder.Replace() method, which does the same thing as String.Replace() but without copying the string in the process. The total memory allocated to hold strings in the preceding code is 150 characters for the StringBuilder instance, as well as the memory allocated during the string operations performed internally in the final Console.WriteLine() statement. Normally, you want to use StringBuilder to perform any manipulation of strings, and String to store or display the final result. StringBuilder Members You have seen a demonstration of one constructor of StringBuilder, which takes an initial string and capacity as its parameters. There are others. For example, you can supply only a string: StringBuilder sb = new StringBuilder("Hello"); Or you can create an empty StringBuilder with a given capacity: StringBuilder sb = new StringBuilder(20); Apart from the Length and Capacity properties, there is a read-only MaxCapacity property that indicates the limit to which a given StringBuilder instance is allowed to grow. By default, this is specified by int.MaxValue (roughly two billion, as noted earlier), but you can set this value to something lower when you construct the StringBuilder object: // This will both set initial capacity to 100, but the max will be 500. // Hence, this StringBuilder can never grow to more than 500 characters, // otherwise it will raise exception if you try to do that. StringBuilder sb = new StringBuilder(100, 500); You can also explicitly set the capacity at any time, though an exception will be raised if you set it to a value less than the current length of the string or a value that exceeds the maximum capacity: c09.indd 214 30-01-2014 20:14:33 Examining System.String  ❘  215 StringBuilder sb = new StringBuilder("Hello"); sb.Capacity = 100; The following table lists the main StringBuilder methods. Method Description Append() Appends a string to the current string. AppendFormat() Appends a string that has been formatted from a format specifier. Insert() Inserts a substring into the current string. Remove() Removes characters from the current string. Replace() Replaces all occurrences of a character with another character or a substring with another substring in the current string. ToString() Returns the current string cast to a System.String object (overridden from System.Object). Several overloads of many of these methods exist. Note  A p p e n d F or m at() is actually the method that is ultimately called when you call Console.WriteLine(), which is responsible for determining what all the format expressions like {0:D} should be replaced with. This method is examined in the next section. There is no cast (either implicit or explicit) from StringBuilder to String. If you want to output the contents of a StringBuilder as a String, you must use the ToString() method. Now that you have been introduced to the StringBuilder class and have learned some of the ways in which you can use it to increase performance, be aware that this class does not always deliver the increased performance you are seeking. Basically, the StringBuilder class should be used when you are manipulating multiple strings. However, if you are just doing something as simple as concatenating two strings, you will find that System.String performs better. Format Strings So far, a large number of classes and structs have been written for the code samples presented in this book, and they have normally implemented a ToString() method in order to display the contents of a given variable. However, users often want the contents of a variable to be displayed in different, often culture- and locale-dependent ways. The .NET base class, System.DateTime, provides the most obvious example of this. For example, you might want to display the same date as 10 June 2012, 10 Jun 2012, 6/10/12 (USA), 10/6/12 (UK), or 10.06.2012 (Germany). Similarly, the Vector struct in Chapter 7, “Operators and Casts,” implements the Vector.ToString() method to display the vector in the format (4, 56, 8). There is, however, another very common way to write vectors, whereby this vector would appear as 4i + 56j + 8k. If you want the classes that you write to be user-friendly, they need to support the capability to display string representations in any of the formats that users are likely to want to use. The .NET runtime defines a standard way in which this should be done: the IFormattable interface. Learning how to add this important feature to your classes and structs is the subject of this section. As you probably know, you need to specify the format in which you want a variable displayed when you call Console.WriteLine(). Therefore, this section uses this method as an example, although most of c09.indd 215 30-01-2014 20:14:33 216  ❘  CHAPTER 9  Strings and Regular Expressions the discussion applies to any situation in which you want to format a string. For example, if you want to display the value of a variable in a list box or text box, you normally use the String.Format() method to obtain the appropriate string representation of the variable. However, the actual format specifiers you use to request a particular format are identical to those passed to Console.WriteLine(). Hence, this section focuses on Console.WriteLine() as an example. It begins by examining what actually happens when you supply a format string to a primitive type, and from this you will see how you can plug format specifiers for your own classes and structs into the process. Chapter 2, “Core C#,” uses format strings in Console.Write() and Console.WriteLine() like this: double d = 13.45; int i = 45; Console.WriteLine("The double is {0,10:E} and the int contains {1}", d, i); The format string itself consists mostly of the text to be displayed; but wherever a variable needs to be formatted, its index in the parameter list appears in braces. You might also include other information inside the braces concerning the format of that item, such as the following: ➤➤ The number of characters to be occupied by the representation of the item, prefixed by a comma. A negative number indicates that the item should be left-justified, whereas a positive number indicates that it should be right-justified. If the item occupies more characters than have been requested, it will still appear in full. ➤➤ A format specifier, preceded by a colon. This indicates how you want the item to be formatted. For example, you can indicate whether you want a number to be formatted as a currency or displayed in scientific notation. The following table lists the common format specifiers for the numeric types, which were briefly discussed in Chapter 2. Specifier Applies To Meaning Example C Numeric types Locale-specific monetary value $4834.50 (USA) £4834.50 (UK) D Integer types only General integer 4834 E Numeric types Scientific notation 4.834E+003 F Numeric types Fixed-point decimal 4384.50 G Numeric types General number 4384.5 N Numeric types Common locale-specific format for numbers 4,384.50 (UK/USA)4 384,50 (continental Europe) P Numeric types Percentage notation 432,000.00% X Integer types only Hexadecimal format 1120 (If you want to display 0x1120, you will have to write out the 0x separately) If you want an integer to be padded with zeros, you can use the format specifier 0 (zero) repeated as many times as the number length requires. For example, the format specifier 0000 will cause 3 to be displayed as 0003, and 99 to be displayed as 0099, and so on. It is not possible to provide a complete list, because other data types can add their own specifiers. The aim here is to demonstrate how to define your own specifiers for your own classes. How the String Is Formatted As an example of how strings are formatted, consider executing the following statement: c09.indd 216 30-01-2014 20:14:34 Examining System.String  ❘  217 Console.WriteLine("The double is {0,10:E} and the int contains {1}", d, i); In the preceding example, Console.WriteLine() just passes the entire set of parameters to the static method, String.Format(). This is the same method that you would call if you wanted to format these values for use in a string to be displayed in a text box, for example. The implementation of the three- parameter overload of WriteLine() basically does this: // Likely implementation of Console.WriteLine() public void WriteLine(string format, object arg0, object arg1) { this.WriteLine(string.Format(this.FormatProvider, format, new object[]{arg0, arg1})); } The one-parameter overload of this method, which is in turn called in the preceding code sample, simply writes out the contents of the string it has been passed, without doing any further formatting on it. String.Format() now needs to construct the final string by replacing each format specifier with a suitable string representation of the corresponding object. However, as shown earlier, for this process of building up a string you need a StringBuilder instance, rather than a string instance. In this example, a StringBuilder instance is created and initialized with the first known portion of the string, the text “The double is”. Next, the StringBuilder.AppendFormat() method is called, passing in the first format specifier, {0,10:E}, as well as the associated object, double, to add the string representation of this object to the string object being constructed. This process continues with StringBuilder.Append() and StringBuilder.AppendFormat() being called repeatedly until the entire formatted string has been obtained. Now the interesting part: StringBuilder.AppendFormat() has to figure out how to format the object. First, it probes the object to determine whether it implements an interface in the System namespace called IFormattable. This can be done quite simply by trying to cast an object to this interface and seeing whether the cast succeeds, or by using the C# is keyword. If this test fails, AppendFormat() calls the object’s ToString() method, which all objects either inherit from System.Object or override. This is exactly what happens here because none of the classes written so far has implemented this interface. That is why the overrides of Object.ToString() have been sufficient to allow the structs and classes from earlier chapters, such as Vector, to be displayed in Console.WriteLine() statements. However, all the predefined primitive numeric types do implement this interface, which means that for those types, and in particular for double and int in the example, the basic ToString() method inherited from System.Object will not be called. To understand what happens instead, you need to examine the IFormattable interface. IFormattable defines just one method, which is also called ToString(). However, this method takes two parameters as opposed to the System.Object version, which doesn’t take any parameters. The following code shows the definition of IFormattable: interface IFormattable { string ToString(string format, IFormatProvider formatProvider); } The first parameter that this overload of ToString() expects is a string that specifies the requested format. In other words, it is the specifier portion of the string that appears inside the braces ({}) in the string originally passed to Console.WriteLine() or String.Format(). For example, in the example the original statement was as follows: Console.WriteLine("The double is {0,10:E} and the int contains {1}", d, i); c09.indd 217 30-01-2014 20:14:34 218  ❘  CHAPTER 9  Strings and Regular Expressions Hence, when evaluating the first specifier, {0,10:E}, this overload is called against the double variable, d, and the first parameter passed to it will be E. The StringBuilder.AppendFormat() method will pass in here the text that appears after the colon in the appropriate format specifier from the original string. We won’t worry about the second ToString() parameter in this book. It is a reference to an object that implements the IFormatProvider interface. This interface provides further information that ToString() might need to consider when formatting the object, such as culture-specific details (a .NET culture is similar to a Windows locale; if you are formatting currencies or dates, you need this information). If you are calling this ToString() overload directly from your source code, you might want to supply such an object. However, StringBuilder.AppendFormat() passes in null for this parameter. If formatProvider is null, then ToString() is expected to use the culture specified in the system settings. Getting back to the example, the first item you want to format is a double, for which you are requesting exponential notation, with the format specifier E. The StringBuilder.AppendFormat() method establishes that the double does implement IFormattable, and will therefore call the two-parameter ToString() overload, passing it the string E for the first parameter and null for the second parameter. It is now up to the double’s implementation of this method to return the string representation of the double in the appropriate format, taking into account the requested format and the current culture. StringBuilder. AppendFormat() will then sort out padding the returned string with spaces, if necessary, to fill the 10 characters specified by the format string. The next object to be formatted is an int, for which you are not requesting any particular format (the format specifier was simply {1}). With no format requested, StringBuilder.AppendFormat() passes in a null reference for the format string. The two-parameter overload of int.ToString() is expected to respond appropriately. No format has been specifically requested; therefore, it calls the no-parameter ToString() method. This entire string formatting process is summarized in Figure 9-2. The FormattableVector Example Now that you know how format strings are constructed, this section extends the Vector example from Chapter 7 so that you can format vectors in a variety of ways. You can download the code for this example from www.wrox.com; the filename is FormattableVector.cs. With your new knowledge of the principles involved now in hand, you will discover that the actual coding is quite simple. All you need to do is implement IFormattable and supply an implementation of the ToString() overload defined by that interface. The format specifiers you are going to support are as follows: ➤➤ N — Should be interpreted as a request to supply a quantity known as the Norm of the Vector. This is just the sum of the squares of its components, which for mathematics buffs happens to be equal to the square of the length of the Vector, and is usually displayed between double vertical bars, like this: ||34.5||. ➤➤ VE — Should be interpreted as a request to display each component in scientific format, just as the specifier E applied to a double indicates (2.3E+01, 4.5E+02, 1.0E+00) ➤➤ IJK — Should be interpreted as a request to display the vector in the form 23i + 450j + 1k ➤➤ Anything else should simply return the default representation of the Vector (23, 450, 1.0). To keep things simple, you are not going to implement any option to display the vector in combined IJK and scientific format. However, you will test the specifier in a case-insensitive way, so that you allow ijk instead of IJK. Note that it is entirely up to you which strings you use to indicate the format specifiers. To achieve this, you first modify the declaration of Vector so it implements IFormattable: c09.indd 218 30-01-2014 20:14:34 Examining System.String  ❘  219 Figure 9-2 RegEx MatchCollection Match GroupCollection Group CaptureCollection Capture Console.WriteLine("The double is {0, 10:E} and the int contains {1}", d, i) String.Format("The double is {0, 10:E} and the int contains {1}", d, i) StringBuilder ("The double is") StringBuilder.AppendFormat ("{0, 10:E}", d) Hello from Wrox! StringBuilder.AppendFormat (" and the int contains ") StringBuilder.AppendFormat ("{1}”, i) c09.indd 219 30-01-2014 20:14:36 220  ❘  CHAPTER 9  Strings and Regular Expressions struct Vector: IFormattable { public double x, y, z; // Beginning part of Vector Now you add your implementation of the two-parameter ToString() overload: public string ToString(string format, IFormatProvider formatProvider) { if (format == null) { return ToString(); } string formatUpper = format.ToUpper(); switch (formatUpper) { case "N": return "|| " + Norm().ToString() + " ||"; case "VE": return String.Format("( {0:E}, {1:E}, {2:E} )", x, y, z); case "IJK": StringBuilder sb = new StringBuilder(x.ToString(), 30); sb.AppendFormat(" i + "); sb.AppendFormat(y.ToString()); sb.AppendFormat(" j + "); sb.AppendFormat(z.ToString()); sb.AppendFormat(" k"); return sb.ToString(); default: return ToString(); } } That is all you have to do! Notice how you take the precaution of checking whether format is null before you call any methods against this parameter — you want this method to be as robust as reasonably possible. The format specifiers for all the primitive types are case insensitive, so that is the behavior that other developers will expect from your class, too. For the format specifier VE, you need each component to be formatted in scientific notation, so you just use String.Format() again to achieve this. The fields x, y, and z are all doubles. For the case of the IJK format specifier, quite a few substrings need to be added to the string, so you use a StringBuilder object to improve performance. For completeness, you also reproduce the no-parameter ToString() overload developed earlier: public override string ToString() { return "( " + x + ", " + y + ", " + z + " )"; } Finally, you need to add a Norm() method that computes the square (norm) of the vector because you didn’t actually supply this method when you developed the Vector struct: public double Norm() { return x*x + y*y + z*z; } Now you can try your formattable vector with some suitable test code: c09.indd 220 30-01-2014 20:14:36 Regular Expressions  ❘  221 static void Main() { Vector v1 = new Vector(1,32,5); Vector v2 = new Vector(845.4, 54.3, -7.8); Console.WriteLine("\nIn IJK format,\nv1 is {0,30:IJK}\nv2 is {1,30:IJK}", v1, v2); Console.WriteLine("\nIn default format,\nv1 is {0,30}\nv2 is {1,30}", v1, v2); Console.WriteLine("\nIn VE format\nv1 is {0,30:VE}\nv2 is {1,30:VE}", v1, v2); Console.WriteLine("\nNorms are:\nv1 is {0,20:N}\nv2 is {1,20:N}", v1, v2); } The result of running this sample is as follows: FormattableVector In IJK format, v1 is 1 i + 32 j + 5 k v2 is 845.4 i + 54.3 j + -7.8 k In default format, v1 is ( 1, 32, 5 ) v2 is ( 845.4, 54.3, -7.8 ) In VE format v1 is ( 1.000000E+000, 3.200000E+001, 5.000000E+000 ) v2 is ( 8.454000E+002, 5.430000E+001, -7.800000E+000 ) Norms are: v1 is || 1050 || v2 is || 717710.49 || This indicates that your custom specifiers are being picked up correctly. Regular Expressions Regular expressions are one of those small technology aids that are incredibly useful in a wide range of programs. You can think of regular expressions as a mini-programming language with one specific purpose: to locate substrings within a large string expression. It is not a new technology; it originated in the UNIX environment and is commonly used with the Perl programming language. Microsoft ported it onto Windows, where up until recently it has been used mostly with scripting languages. Today, regular expressions are supported by a number of .NET classes in the namespace System.Text .RegularExpressions. You can also find the use of regular expressions in various parts of the .NET Framework. For instance, they are used within the ASP.NET validation server controls. If you are not familiar with the regular expressions language, this section introduces both regular expressions and their related .NET classes. If you are familiar with regular expressions, you will probably want to just skim through this section to pick out the references to the .NET base classes. You might like to know that the .NET regular expression engine is designed to be mostly compatible with Perl 5 regular expressions, although it has a few extra features. Introduction to Regular Expressions The regular expressions language is designed specifically for string processing. It contains two features: A set of escape codes for identifying specific types of characters. You will be familiar with the use of the * character to represent any substring in DOS expressions. (For example, the DOS command Dir Re* lists the files with names beginning with Re.) Regular expressions use many sequences like this to represent items such as any one character, a word break, one optional character, and so on. ➤➤ A system for grouping parts of substrings and intermediate results during a search operation c09.indd 221 30-01-2014 20:14:36 222  ❘  CHAPTER 9  Strings and Regular Expressions With regular expressions, you can perform very sophisticated and high-level operations on strings. For example, you can do all of the following: ➤➤ Identify (and perhaps either flag or remove) all repeated words in a string (e.g., “The computer books books” to “The computer books”) ➤➤ Convert all words to title case (e.g., “this is a Title” to “This Is A Title”) ➤➤ Convert all words longer than three characters to title case (e.g., “this is a Title” to “This is a Title”) ➤➤ Ensure that sentences are properly capitalized ➤➤ Separate the various elements of a URI (e.g., given http://www.wrox.com, extract the protocol, computer name, filename, and so on) Of course, all these tasks can be performed in C# using the various methods on System.String and System.Text.StringBuilder. However, in some cases, this would require writing a fair amount of C# code. Using regular expressions, this code can normally be compressed to just a couple of lines. Essentially, you instantiate a System.Text.RegularExpressions.RegEx object (or, even simpler, invoke a static RegEx() method), pass it the string to be processed, and pass in a regular expression (a string containing the instructions in the regular expressions language), and you’re done. A regular expression string looks at first sight rather like a regular string, but interspersed with escape sequences and other characters that have a special meaning. For example, the sequence \b indicates the beginning or end of a word (a word boundary), so if you wanted to indicate you were looking for the characters th at the beginning of a word, you would search for the regular expression, \bth (that is, the sequence word boundary-t-h). If you wanted to search for all occurrences of th at the end of a word, you would write th\b (the sequence t-h-word boundary). However, regular expressions are much more sophisticated than that and include, for example, facilities to store portions of text that are found in a search operation. This section only scratches the surface of the power of regular expressions. Note  For more on regular expressions, please see Andrew Watt’s Beginning Regular Expressions (John Wiley & Sons, 2005). Suppose your application needed to convert U.S. phone numbers to an international format. In the United States, the phone numbers have the format 314-123-1234, which is often written as (314) 123-1234. When converting this national format to an international format, you have to include +1 (the country code of the United States) and add brackets around the area code: +1 (314) 123-1234. As find-and-replace operations go, that is not too complicated. It would still require some coding effort if you were going to use the String class for this purpose (meaning you would have to write your code using the methods available from System.String). The regular expressions language enables you to construct a short string that achieves the same result. This section is intended only as a very simple example, so it concentrates on searching strings to identify certain substrings, not on modifying them. The RegularExpressionsPlayaround Example The rest of this section develops a short example called RegularExpressionsPlayaround that illustrates some of the features of regular expressions, and how to use the .NET regular expressions engine in C# by performing and displaying the results of some searches. The text you are going to use as your sample document is the introduction to a book on ASP.NET, Professional ASP.NET 4: in C# and VB (Wiley, 2010): const string myText = @"This comprehensive compendium provides a broad and thorough investigation of all aspects of programming with ASP.NET. Entirely revised and updated for the fourth release of .NET, this book will give you the information you need to master ASP.NET and build a dynamic, successful, enterprise Web application."; c09.indd 222 30-01-2014 20:14:36 Regular Expressions  ❘  223 Note  This code is valid C# code, despite all the line breaks. It nicely ­illustrates the utility of verbatim strings that are prefixed by the @ symbol. This text is referred to as the input string. To get your bearings and get used to the regular expressions of .NET classes, you start with a basic plain -text search that does not feature any escape sequences or regular expression commands. Suppose that you want to find all occurrences of the string “ion”. This search string is referred to as the pattern. Using regular expressions and the Text variable declared previously, you could write the following: const string pattern = "ion"; MatchCollection myMatches = Regex.Matches(myText, pattern, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture); foreach (Match nextMatch in myMatches) { Console.WriteLine(nextMatch.Index); } This code uses the static method Matches() of the Regex class in the System.Text.RegularExpressions namespace. This method takes as parameters some input text, a pattern, and a set of optional flags taken from the RegexOptions enumeration. In this case, you have specified that all searching should be caseinsensitive. The other flag, ExplicitCapture, modifies how the match is collected in a way that, for your purposes, makes the search a bit more efficient — you’ll see why this is later (although it does have other uses that we won’t explore here). Matches() returns a reference to a MatchCollection object. A match is the technical term for the results of finding an instance of the pattern in the expression. It is represented by the class System.Text.RegularExpressions.Match. Therefore, you return a MatchCollection that contains all the matches, each represented by a Match object. In the preceding code, you simply iterate over the collection and use the Index property of the Match class, which returns the index in the input text where the match was found. Running this code results in three matches. The following table details some of the RegexOptions enumerations. Member Name Description CultureInvariant Specifies that the culture of the string is ignored. ExplicitCapture Modifies the way the match is collected by making sure that valid captures are the ones that are explicitly named. IgnoreCase Ignores the case of the string that is input. IgnorePatternWhitespace Removes unescaped whitespace from the string and enables comments that are specified with the pound or hash sign. Multiline Changes the characters ^ and $ so that they are applied to the beginning and end of each line and not just to the beginning and end of the entire string. RightToLeft Causes the inputted string to be read from right to left instead of the default left to right (ideal for some Asian and other languages that are read in this direction). Singleline Specifies a single-line mode where the meaning of the dot (.) is changed to match every character. So far, nothing is new from the preceding example apart from some .NET base classes. However, the power of regular expressions comes from that pattern string. The reason is because the pattern string is not limited to only plain text. As hinted earlier, it can also contain what are known as meta-characters, which are c09.indd 223 30-01-2014 20:14:36 224  ❘  CHAPTER 9  Strings and Regular Expressions special characters that provide commands, as well as escape sequences, which work in much the same way as C# escape sequences. They are characters preceded by a backslash (\) and have special meanings. For example, suppose you wanted to find words beginning with n. You could use the escape sequence \b, which indicates a word boundary (a word boundary is just a point where an alphanumeric character precedes or follows a whitespace character or punctuation symbol): const string pattern = @"\bn"; MatchCollection myMatches = Regex.Matches(myText, pattern, RegexOptions.IgnoreCase | RegexOptions.ExplicitCapture); Notice the @ character in front of the string. You want the \b to be passed to the .NET regular expressions engine at runtime — you don’t want the backslash intercepted by a well-meaning C# compiler that thinks it’s an escape sequence in your source code. If you want to find words ending with the sequence ion, you write this: const string pattern = @"ion\b"; If you want to find all words beginning with the letter a and ending with the sequence ion (which has as its only match the word application in the example), you have to put a bit more thought into your code. You clearly need a pattern that begins with \ba and ends with ion\b, but what goes in the middle? You need to somehow tell the application that between the a and the ion there can be any number of characters as long as none of them are whitespace. In fact, the correct pattern looks like this: const string pattern = @"\ba\S*ion\b"; Eventually you will get used to seeing weird sequences of characters like this when working with regular expressions. It actually works quite logically. The escape sequence \S indicates any character that is not a whitespace character. The * is called a quantifier. It means that the preceding character can be repeated any number of times, including zero times. The sequence \S* means any number of characters as long as they are not whitespace characters. The preceding pattern will, therefore, match any single word that begins with a and ends with ion. The following table lists some of the main special characters or escape sequences that you can use. It is not comprehensive, but a fuller list is available in the MSDN documentation. Symbol Description Example Matches ^ Beginning of input text ^B B, but only if first character in text $ End of input text X$ X, but only if last character in text . Any single character except the newline character (\ ) i.ation isation, ization * Preceding character may be repeated zero or more times ra*t rt, rat, raat, raaat, and so on + Preceding character may be repeated one or more times ra+t rat, raat, raaat and so on, but not rt ? Preceding character may be repeated zero or one time ra?t rt and rat only \s Any whitespace character \sa [space]a, \ta, \na (\t and \n have the same meanings as in C#) \S Any character that isn’t whitespace \SF aF, rF, cF, but not \tf \b Word boundary ion\b Any word ending in ion \B Any position that isn’t a word boundary \BX\B Any X in the middle of a word c09.indd 224 30-01-2014 20:14:37 Regular Expressions  ❘  225 If you want to search for one of the meta-characters, you can do so by escaping the corresponding character with a backslash. For example, . (a single period) means any single character other than the newline character, whereas \. means a dot. You can request a match that contains alternative characters by enclosing them in square brackets. For example, [1|c] means one character that can be either 1 or c. If you wanted to search for any occurrence of the words map or man, you would use the sequence ma[n|p]. Within the square brackets, you can also indicate a range, for example [a-z], to indicate any single lowercase letter, [A-E] to indicate any uppercase letter between A and E (including the letters A and E themselves), or [0–9] to represent a single digit. If you wanted to search for an integer (that is, a sequence that contains only the characters 0 through 9), you could write [0–9]+. Note  The use of the + character specifies there must be at least one such digit, but there may be more than one — so this would match 9, 83, 854, and so on. Displaying Results In this section, you code the RegularExpressionsPlayaround example to get a feel for how regular expressions work. The core of the example is a method called WriteMatches(), which writes out all the matches from a MatchCollection in a more detailed format. For each match, it displays the index of where the match was found in the input string, the string of the match, and a slightly longer string, which consists of the match plus up to 10 surrounding characters from the input text — up to five characters before the match and up to five afterward. (It is fewer than five characters if the match occurred within five characters of the beginning or end of the input text.) In other words, a match on the word messaging that occurs near the end of the input text quoted earlier would display and messaging of d (five characters before and after the match), but a match on the final word data would display g of data. (only one character after the match), because after that you get to the end of the string. This longer string enables you to see more clearly where the regular expression locates the match: static void WriteMatches(string text, MatchCollection matches) { Console.WriteLine("Original text was: \n\n" + text + "\n"); Console.WriteLine("No. of matches: " + matches.Count); foreach (Match nextMatch in matches) { int index = nextMatch.Index; string result = nextMatch.ToString(); int charsBefore = (index < 5) ? index: 5; int fromEnd = text.Length-index-result.Length; int charsAfter = (fromEnd < 5) ? fromEnd: 5; int charsToDisplay = charsBefore + charsAfter + result.Length; Console.WriteLine("Index: {0}, \tString: {1}, \t{2}", index, result, text.Substring(index-charsBefore, charsToDisplay)); } } The bulk of the processing in this method is devoted to the logic of figuring out how many characters in the longer substring it can display without overrunning the beginning or end of the input text. Note that you use another property on the Match object, Value, which contains the string identified for the match. Other than that, RegularExpressionsPlayaround simply contains a number of methods with names such as c09.indd 225 30-01-2014 20:14:37 226  ❘  CHAPTER 9  Strings and Regular Expressions Find1, Find2, and so on, which perform some of the searches based on the examples in this section. For example, Find2 looks for any string that contains a at the beginning of a word: static void Find2() { string text = @"This comprehensive compendium provides a broad and thorough investigation of all aspects of programming with ASP.NET. Entirely revised and updated for the 3.5 Release of .NET, this book will give you the information you need to master ASP.NET and build a dynamic, successful, enterprise Web application."; string pattern = @"\ba"; MatchCollection matches = Regex.Matches(text, pattern, RegexOptions.IgnoreCase); WriteMatches(text, matches); } Along with this is a simple Main() method that you can edit to select one of the Find() methods: static void Main() { Find1(); Console.ReadLine(); } The code also needs to make use of the RegularExpressions namespace: using System; using System.Text.RegularExpressions; Running the example with the Find2() method shown previously gives these results: RegularExpressionsPlayaround Original text was: This comprehensive compendium provides a broad and thorough investigation of all aspects of programming with ASP.NET. Entirely revised and updated for the 3.5 Release of .NET, this book will give you the information you need to master ASP.NET and build a dynamic, successful, enterprise Web application. No. of matches: 1 Index: 291, String: application, Web application. Matches, Groups, and Captures One nice feature of regular expressions is that you can group characters. It works the same way as compound statements in C#. In C#, you can group any number of statements by putting them in braces, and the result is treated as one compound statement. In regular expression patterns, you can group any characters (including meta-characters and escape sequences), and the result is treated as a single character. The only difference is that you use parentheses instead of braces. The resultant sequence is known as a group. For example, the pattern (an)+ locates any recurrences of the sequence an. The + quantifier applies only to the previous character, but because you have grouped the characters together, it now applies to repeats of an treated as a unit. This means that if you apply (an)+ to the input text, bananas came to Europe late in the annals of history, the anan from bananas is identified; however, if you write an+, the program selects the ann from annals, as well as two separate sequences of an from bananas. The expression (an)+ c09.indd 226 30-01-2014 20:14:37 Regular Expressions  ❘  227 identifies occurrences of an, anan, ananan, and so on, whereas the expression an+ identifies occurrences of an, ann, annn, and so on. Note  You might be wondering why with the preceding example (a n)+ selects anan from the word “banana” but doesn’t identify either of the two occurrences of an from the same word. The rule is that matches must not overlap. If a couple of possibilities would overlap, then by default the longest possible sequence is matched. However, groups are actually more powerful than that. By default, when you form part of the pattern into a group, you are also asking the regular expression engine to remember any matches against just that group, as well as any matches against the entire pattern. In other words, you are treating that group as a pattern to be matched and returned in its own right. This can be extremely useful if you want to break up strings into component parts. For example, URIs have the format ://
:, where the port is optional. An example of this is http://www.wrox.com:4355. Suppose you want to extract the protocol, the address, and the port from a URI in which there may or may not be whitespace (but no punctuation) immediately following the URI. You could do so using this expression: \b(\S+)://([^:]+)(?::(\S+))?\b Here is how this expression works: First, the leading and trailing \b sequences ensure that you consider only portions of text that are entire words. Within that, the first group, (\S+)://, identifies one or more characters that don’t count as whitespace, and that are followed by:// — the http:// at the start of an HTTP URI. The brackets cause the http to be stored as a group. Next, ([^:]+) identifies the string www.wrox.com in the URI. This group will end either when it encounters the end of the word (the closing \b) or a colon (:) as marked by the next group. The next group identifies the port (:4355). The following ? indicates that this group is optional in the match — if there is no: xxxx, this won’t prevent a match from being marked. This is very important because the port number is not always specified in a URI — in fact, it is usually absent. However, things are a bit more complicated than that. You want to indicate that the colon might or might not appear too, but you don’t want to store this colon in the group. You achieved this by using two nested groups. The inner (\S+) identifies anything that follows the colon (for example, 4355). The outer group contains the inner group preceded by the colon, and this group in turn is preceded by the sequence ?:. This sequence indicates that the group in question should not be saved (you only want to save 4355; you don’t need :4355 as well!). Don’t be confused by the two colons following each other — the first colon is part of the ?: sequence that says “don’t save this group,” and the second is text to be searched for. If you run this pattern on the following string, you’ll get one match: http://www.wrox.com: Hey I've just found this amazing URI at http:// what was it --oh yes http://www.wrox.com Within this match you will find the three groups just mentioned, as well as a fourth group that represents the match itself. Theoretically, it is possible for each group itself to return no, one, or more than one match. Each of these individual matches is known as a capture. Therefore, the first group, (\S+), has one capture, http. The second group also has one capture (www.wrox.com). The third group, however, has no captures, because there is no port number on this URI. Notice that the string contains a second http://. Although this does match up to the first group, it will not be captured by the search because the entire search expression does not match this part of the text. There isn’t space here to show examples of C# code that uses groups and captures, but you should know that the .NET RegularExpressions classes support groups and captures through classes known as Group and c09.indd 227 30-01-2014 20:14:37 228  ❘  CHAPTER 9  Strings and Regular Expressions RegEx MatchCollection Match GroupCollection Group CaptureCollection Capture Figure 9-3 Capture. Also, the GroupCollection and CaptureCollection classes represent collections of groups and captures, respectively. The Match class exposes the Groups property, which returns the corresponding GroupCollection object. The Group class correspondingly implements the Captures property, which returns a CaptureCollection. The relationship between the objects is shown in Figure 9-3. You might not want to return a Group object every time you just want to group some characters. A fair amount of overhead is involved in instantiating the object, which is not necessary if all you want to do is group some characters as part of your search pattern. You can disable this by starting the group with the character sequence ?: for an individual group, as was done for the URI example, or for all groups by specifying the RegExOptions.ExplicitCaptures flag on the RegEx.Matches() method, as was done in the earlier examples. Summary You have quite a number of available data types at your disposal when working with the .NET Framework. One of the most frequently used types in your applications (especially applications that focus on submitting and retrieving data) is the string data type. The importance of string is the reason why this book has an entire chapter that focuses on how to use the string data type and manipulate it in your applications. When working with strings in the past, it was quite common to just slice and dice the strings as needed using concatenation. With the .NET Framework, you can use the StringBuilder class to accomplish a lot of this task with better performance than before. Last, but hardly least, advanced string manipulation using regular expressions is an excellent tool to search through and validate your strings. c09.indd 228 30-01-2014 20:14:39 Collections wHAT’s iN THis CHAPTER? ➤➤ Understanding collection interfaces and types ➤➤ Working with lists, queues, and stacks ➤➤ Working with linked and sorted lists ➤➤ Using dictionaries and sets ➤➤ Using bit arrays and bit vectors ➤➤ Using immutable and concurrent collections ➤➤ Evaluating performance wROX.COM COdE dOwNlOAds fOR THis CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ List Samples ➤➤ Queue Sample ➤➤ Linked List Sample ➤➤ Sorted List Sample ➤➤ Dictionary Sample ➤➤ Set Sample ➤➤ Observable Collection Sample ➤➤ BitArray Sample ➤➤ Immutable Collections Sample ➤➤ Pipeline Sample 10 c10.indd 229 30-01-2014 20:15:17 230  ❘  CHAPTER 10  Collections Overview In Chapter 6, “Arrays and Tuples,” you learned about arrays and the interfaces implemented by the Array class. The size of arrays is fixed. If the number of elements is dynamic, you should use a collection class instead of an array. List is a collection class that can be compared to arrays; but there are also other kinds of collections: queues, stacks, linked lists, dictionaries, and sets. The other collection classes have partly different APIs to access the elements in the collection and often a different internal structure for how the items are stored in memory. This chapter covers all of these collection classes and their differences, including performance differences. You can also read about bit arrays and concurrent collections that can be used from multiple threads. Note  Version 1 of the .NET Framework included only non-generic collection classes such as ArrayList and HashTable. CLR 2.0 added support for generics and added generic collection classes. The focus of this chapter is just on the newer group of collection classes and ignores the old ones, as they are not needed with new applications. Collection Interfaces and Types Most collection classes can be found in the System.Collections and System.Collections.Generic namespaces. Generic collection classes are located in the System.Collections.Generic namespace. Collection classes that are specialized for a specific type are located in the System.Collections .Specialized namespace. Thread-safe collection classes are in the System.Collections.Concurrent namespace. Immutable collection classes are in the System.Collections.Immutable namespace. Of course, there are also other ways to group collection classes. Collections can be grouped into lists, collec- tions, and dictionaries based on the interfaces that are implemented by the collection class. Note  You can read detailed information about the interfaces IEnumerable and IEnumerator in Chapter 6. The following table describes interfaces implemented by collections and lists. Interface Description IEnumerable The interface IEnumerable is required by the foreach statement. This interface defines the method GetEnumerator which returns an enumerator that implements the IEnumerator interface. ICollection ICollection is implemented by generic collection classes. With this you can get the number of items in the collection (Count property), and copy the collection to an array (CopyTo method). You can also add and remove items from the collection (Add, Remove, Clear). c10.indd 230 30-01-2014 20:15:18 Lists  ❘  231 IList The IList interface is for lists where elements can be accessed from their position. This interface defines an indexer, as well as ways to insert or remove items from specific positions (Insert, RemoveAt methods). IList derives from ICollection. ISet This interface is implemented by sets. Sets allow combin- ing different sets into a union, getting the intersection of two sets, and checking whether two sets overlap. ISet derives from ICollection. IDictionary The interface IDictionary is implement- ed by generic collection classes that have a key and a value. With this interface all the keys and values can be accessed, items can be accessed with an indexer of type key, and items can be added or removed. ILookup Similar to the IDictionary interface, lookups have keys and values. However, with lookups the collection can contain multiple values with one key. IComparer The interface IComparer is implemented by a com- parer and used to sort elements inside a collection with the Compare method. IEqualityComparer IEqualityComparer is implemented by a comparer that can be used for keys in a dictionary. With this interface the objects can be compared for equality. Since .NET 4, this interface is also implemented by arrays and tuples. IProducerConsumerCollection The interface IProducerConsumerCollection is new since .NET 4 and supports new thread-safe collection classes. IReadOnlyCollection IReadOnlyList IReadOnlyDictionary The interfaces IReadOnlyCollection, IReadOnlyList, and IReadOnlyDictionary are for collections that cannot be changed after initialization. The members of these interfaces allow only retrieving objects, but not adding or modifying them. IImmutableArray IImmutableList IImmutableQueue IImmutableSet IImmutableDictionary The immutable interfaces define methods and properties for immutable collections. These collections cannot be changed after initialization. Lists For resizable lists, the .NET Framework offers the generic class List. This class implements the IList, ICollection, IEnumerable, IList, ICollection, and IEnumerable interfaces. The following examples use the members of the class Racer as elements to be added to the collection to rep- resent a Formula-1 racer. This class has five properties: Id, FirstName, LastName, Country, and the num- ber of Wins. With the constructors of the class, the name of the racer and the number of wins can be passed to set the members. The method ToString is overridden to return the name of the racer. The class Racer c10.indd 231 30-01-2014 20:15:18 232  ❘  CHAPTER 10  Collections also implements the generic interface IComparable for sorting racer elements and IFormattable (code file ListSamples/Racer.cs): [Serializable] public class Racer: IComparable, IFormattable { public int Id { get; private set; } public string FirstName { get; set; } public string LastName { get; set; } public string Country { get; set; } public int Wins { get; set; } public Racer(int id, string firstName, string lastName, string country) :this(id, firstName, lastName, country, wins: 0) { } public Racer(int id, string firstName, string lastName, string country, int wins) { this.Id = id; this.FirstName = firstName; this.LastName = lastName; this.Country = country; this.Wins = wins; } public override string ToString() { return String.Format("{0} {1}", FirstName, LastName); } public string ToString(string format, IFormatProvider formatProvider) { if (format == null) format = "N"; switch (format.ToUpper()) { case "N": // name return ToString(); case "F": // first name return FirstName; case "L": // last name return LastName; case "W": // Wins return String.Format("{0}, Wins: {1}", ToString(), Wins); case "C": // Country return String.Format("{0}, Country: {1}", ToString(), Country); case "A": // All return String.Format("{0}, {1} Wins: {2}", ToString(), Country, Wins); default: throw new FormatException(String.Format(formatProvider, "Format {0} is not supported", format)); } } public string ToString(string format) { c10.indd 232 30-01-2014 20:15:18 Lists  ❘  233 return ToString(format, null); } public int CompareTo(Racer other) { if (other == null) return -1; int compare = string.Compare(this.LastName, other.LastName); if (compare == 0) return string.Compare(this.FirstName, other.FirstName); return compare; } } Creating Lists You can create list objects by invoking the default constructor. With the generic class List, you must specify the type for the values of the list with the declaration. The following code shows how to declare a List with int and a list with Racer elements. ArrayList is a non-generic list that accepts any Object type for its elements. Using the default constructor creates an empty list. As soon as elements are added to the list, the capacity of the list is extended to allow four elements. If the fifth element is added, the list is resized to allow eight ele- ments. If eight elements are not enough, the list is resized again to contain 16 elements. With every resize the capacity of the list is doubled: var intList = new List(); var racers = new List(); If the capacity of the list changes, the complete collection is reallocated to a new memory block. With the implementation of List, an array of type T is used. With reallocation, a new array is created, and Array.Copy copies the elements from the old array to the new array. To save time, if you know the number of elements in advance that should be in the list; you can define the capacity with the constructor. The fol- lowing example creates a collection with a capacity of 10 elements. If the capacity is not large enough for the elements added, the capacity is resized to 20 and then to 40 elements — doubled again: List intList = new List(10); You can get and set the capacity of a collection by using the Capacity property: intList.Capacity = 20; The capacity is not the same as the number of elements in the collection. The number of elements in the col- lection can be read with the Count property. Of course, the capacity is always larger or equal to the number of items. As long as no element was added to the list, the count is 0: Console.WriteLine(intList.Count); If you are finished adding elements to the list and don’t want to add any more, you can get rid of the unneeded capacity by invoking the TrimExcess method; however, because the relocation takes time, TrimExcess has no effect if the item count is more than 90 percent of capacity: intList.TrimExcess(); Collection Initializers You can also assign values to collections using collection initializers. The syntax of collection initializers is similar to array initializers, explained in Chapter 6. With a collection initializer, values are assigned to the collection within curly brackets at the time the collection is initialized: var intList = new List() {1, 2}; var stringList = new List() {"one", "two"}; c10.indd 233 30-01-2014 20:15:18 234  ❘  CHAPTER 10  Collections Note  Collection initializers are not reflected within the IL code of the compiled assembly. The compiler converts the collection initializer to invoke the Add method for every item from the initializer list. Adding Elements You can add elements to the list with the Add method, shown in the following example. The generic instanti- ated type defines the parameter type of the Add method: var intList = new List(); intList.Add(1); intList.Add(2); var stringList = new List(); stringList.Add("one"); stringList.Add("two"); The variable racers is defined as type List. With the new operator, a new object of the same type is created. Because the class List was instantiated with the concrete class Racer, now only Racer objects can be added with the Add method. In the following sample code, five Formula-1 racers are created and added to the collection. The first three are added using the collection initializer, and the last two are added by invoking the Add method explicitly (code file ListSamples/Program.cs): var graham = new Racer(7, "Graham", "Hill", "UK", 14); var emerson = new Racer(13, "Emerson", "Fittipaldi", "Brazil", 14); var mario = new Racer(16, "Mario", "Andretti", "USA", 12); var racers = new List(20) {graham, emerson, mario}; racers.Add(new Racer(24, "Michael", "Schumacher", "Germany", 91)); racers.Add(new Racer(27, "Mika", "Hakkinen", "Finland", 20)); With the AddRange method of the List class, you can add multiple elements to the collection at once. The method AddRange accepts an object of type IEnumerable, so you can also pass an array as shown here: racers.AddRange(new Racer[] { new Racer(14, "Niki", "Lauda", "Austria", 25), new Racer(21, "Alain", "Prost", "France", 51)}); Note  The collection initializer can be used only during declaration of the collection. The AddRange method can be invoked after the collection is initialized. In case you get the data dynamically after creating the collection, you need to invoke AddRange. If you know some elements of the collection when instantiating the list, you can also pass any object that implements IEnumerable to the constructor of the class. This is very similar to the AddRange method: var racers = new List( new Racer[] { new Racer(12, "Jochen", "Rindt", "Austria", 6), new Racer(22, "Ayrton", "Senna", "Brazil", 41) }); Inserting Elements You can insert elements at a specified position with the Insert method: racers.Insert(3, new Racer(6, "Phil", "Hill", "USA", 3)); c10.indd 234 30-01-2014 20:15:18 Lists  ❘  235 The method InsertRange offers the capability to insert a number of elements, similar to the AddRange method shown earlier. If the index set is larger than the number of elements in the collection, an exception of type ArgumentOutOfRangeException is thrown. Accessing Elements All classes that implement the IList and IList interface offer an indexer, so you can access the ele- ments by using an indexer and passing the item number. The first item can be accessed with an index value 0. By specifying racers[3], for example, you access the fourth element of the list: Racer r1 = racers[3]; Getting the number of elements with the Count property, you can do a for loop to iterate through every item in the collection, and use the indexer to access every item: for (int i = 0; i < racers.Count; i++) { Console.WriteLine(racers[i]); } Note  Indexed access to collection classes is available with ArrayList, StringCollection, and List. Because List implements the interface IEnumerable, you can iterate through the items in the collection using the foreach statement as well: foreach (Racer r in racers) { Console.WriteLine(r); } Note  How the foreach statement is resolved by the compiler to make use of the IEnumerable and IEnumerator interfaces, is explained in Chapter 6. Instead of using the foreach statement, the List class also offers a ForEach method that is declared with an Action parameter: public void ForEach(Action action); The implementation of ForEach is shown next. ForEach iterates through every item of the collection and invokes the method that is passed as a parameter for every item: public class List: IList { private T[] items; //... public void ForEach(Action action) { if (action == null) throw new ArgumentNullException("action"); foreach (T item in items) { c10.indd 235 30-01-2014 20:15:19 236  ❘  CHAPTER 10  Collections action(item); } } //... } To pass a method with ForEach, Action is declared as a delegate that defines a method with a void return type and parameter T: public delegate void Action(T obj); With a list of Racer items, the handler for the ForEach method must be declared with a Racer object as a parameter and a void return type: public void ActionHandler(Racer obj); Because one overload of the Console.WriteLine method accepts Object as a parameter, you can pass the address of this method to the ForEach method, and every racer of the collection is written to the console: racers.ForEach(Console.WriteLine); You can also write a lambda expression that accepts a Racer object as a parameter and contains an imple- mentation to write a string to the console using Console.WriteLine. Here, the format A is used with the ToString method of the IFormattable interface to display all information about the racer: racers.ForEach(r => Console.WriteLine("{0:A}", r)); Note  Lambda expressions are explained in Chapter 8, “Delegates, Lambdas, and Events.” Removing Elements You can remove elements by index or pass the item that should be removed. Here, the fourth element is removed from the collection: racers.RemoveAt(3); You can also directly pass a Racer object to the Remove method to remove this element. Removing by index is faster, because here the collection must be searched for the item to remove. The Remove method first searches in the collection to get the index of the item with the IndexOf method, and then uses the index to remove the item. IndexOf first checks if the item type implements the interface IEquatable. If it does, the Equals method of this interface is invoked to find the item in the collection that is the same as the one passed to the method. If this interface is not implemented, the Equals method of the Object class is used to compare the items. The default implementation of the Equals method in the Object class does a bitwise comparison with value types, but compares only references with reference types. Note  Chapter 7, “Operators and Casts,” explains how you can override the Equals method. In the following example, the racer referenced by the variable graham is removed from the collection. The variable graham was created earlier when the collection was filled. Because the interface IEquatable and the Object.Equals method are not overridden with the Racer class, you cannot create a new object with the same content as the item that should be removed and pass it to the Remove method: if (!racers.Remove(graham)) { Console.WriteLine("object not found in collection"); } c10.indd 236 30-01-2014 20:15:19 Lists  ❘  237 The method RemoveRange removes a number of items from the collection. The first parameter specifies the index where the removal of items should begin; the second parameter specifies the number of items to be removed: int index = 3; int count = 5; racers.RemoveRange(index, count); To remove all items with some specific characteristics from the collection, you can use the RemoveAll method. This method uses the Predicate parameter when searching for elements, which is discussed next. To remove all elements from the collection, use the Clear method defined with the ICollection interface. Searching There are different ways to search for elements in the collection. You can get the index to the found item, or the item itself. You can use methods such as IndexOf, LastIndexOf, FindIndex, FindLastIndex, Find, and FindLast. To just check whether an item exists, the List class offers the Exists method. The method IndexOf requires an object as a parameter and returns the index of the item if it is found inside the collection. If the item is not found, –1 is returned. Remember that IndexOf is using the IEquatable interface to compare the elements: int index1 = racers.IndexOf(mario); With the IndexOf method, you can also specify that the complete collection should not be searched, instead specifying an index where the search should start and the number of elements that should be iterated for the comparison. Instead of searching a specific item with the IndexOf method, you can search for an item that has some specific characteristics that you can define with the FindIndex method. FindIndex requires a parameter of type Predicate: public int FindIndex(Predicate match); The Predicate type is a delegate that returns a Boolean value and requires type T as a parameter. This delegate can be used similarly to the Action delegate shown earlier with the ForEach method. If the predi- cate returns true, there’s a match and the element is found. If it returns false, the element is not found and the search continues: public delegate bool Predicate(T obj); With the List class that is using Racer objects for type T, you can pass the address of a method that returns a bool and defines a parameter of type Racer to the FindIndex method. Finding the first racer of a specific country, you can create the FindCountry class as shown next. The FindCountryPredicate method has the signature and return type defined by the Predicate delegate. The Find method uses the variable country to search for a country that you can pass with the constructor of the class: public class FindCountry { public FindCountry(string country) { this.country = country; } private string country; public bool FindCountryPredicate(Racer racer) { Contract.Requires(racer != null); return racer.Country == country; } } c10.indd 237 30-01-2014 20:15:19 238  ❘  CHAPTER 10  Collections With the FindIndex method, you can create a new instance of the FindCountry class, pass a country string to the constructor, and pass the address of the Find method. In the following example, after FindIndex completes successfully, index2 contains the index of the first item where the Country property of the racer is set to Finland: int index2 = racers.FindIndex(new FindCountry("Finland"). FindCountryPredicate); Instead of creating a class with a handler method, you can use a lambda expression here as well. The result is exactly the same as before. Now the lambda expression defines the implementation to search for an item where the Country property is set to Finland: int index3 = racers.FindIndex(r => r.Country == "Finland"); Similar to the IndexOf method, with the FindIndex method you can also specify the index where the search should start and the count of items that should be iterated through. To do a search for an index beginning from the last element in the collection, you can use the FindLastIndex method. The method FindIndex returns the index of the found item. Instead of getting the index, you can also go directly to the item in the collection. The Find method requires a parameter of type Predicate, much as the FindIndex method. The Find method in the following example searches for the first racer in the list that has the FirstName property set to Niki. Of course, you can also do a FindLast search to find the last item that fulfills the predicate: Racer racer = racers.Find(r => r.FirstName == "Niki"); To get not only one, but all the items that fulfill the requirements of a predicate, you can use the FindAll method. The FindAll method uses the same Predicate delegate as the Find and FindIndex methods. The FindAll method does not stop when the first item is found but instead iterates through every item in the collection and returns all items for which the predicate returns true. With the FindAll method invoked in the next example, all racer items are returned where the property Wins is set to more than 20. All racers who won more than 20 races are referenced from the bigWinners list: List bigWinners = racers.FindAll(r => r.Wins > 20); Iterating through the variable bigWinners with a foreach statement gives the following result: foreach (Racer r in bigWinners) { Console.WriteLine("{0:A}", r); } Michael Schumacher, Germany Wins: 91 Niki Lauda, Austria Wins: 25 Alain Prost, France Wins: 51 The result is not sorted, but you’ll see that done next. Sorting The List class enables sorting its elements by using the Sort method. Sort uses the quick sort algo- rithm whereby all elements are compared until the complete list is sorted. You can use several overloads of the Sort method. The arguments that can be passed are a generic del- egate Comparison, the generic interface IComparer, and a range together with the generic interface IComparer: public void List.Sort(); public void List.Sort(Comparison); public void List.Sort(IComparer); public void List.Sort(Int32, Int32, IComparer); c10.indd 238 30-01-2014 20:15:19 Lists  ❘  239 Using the Sort method without arguments is possible only if the elements in the collection implement the interface IComparable. Here, the class Racer implements the interface IComparable to sort racers by last name: racers.Sort(); racers.ForEach(Console.WriteLine); If you need to do a sort other than the default supported by the item types, you need to use other techniques, such as passing an object that implements the IComparer interface. The class RacerComparer implements the interface IComparer for Racer types. This class enables you to sort by either the first name, last name, country, or number of wins. The kind of sort that should be done is defined with the inner enumeration type CompareType. The CompareType is set with the constructor of the class RacerComparer. The interface IComparer defines the method Compare, which is required for sorting. In the implementation of this method, the Compare and CompareTo methods of the string and int types are used (code file ListSamples/RacerComparer.cs): public class RacerComparer: IComparer { public enum CompareType { FirstName, LastName, Country, Wins } private CompareType compareType; public RacerComparer(CompareType compareType) { this.compareType = compareType; } public int Compare(Racer x, Racer y) { if (x == null && y == null) return 0; if (x == null) return -1; if (y == null) return 1; int result; switch (compareType) { case CompareType.FirstName: return string.Compare(x.FirstName, y.FirstName); case CompareType.LastName: return string.Compare(x.LastName, y.LastName); case CompareType.Country: result = string.Compare(x.Country, y.Country); if (result == 0) return string.Compare(x.LastName, y.LastName); else return result; case CompareType.Wins: return x.Wins.CompareTo(y.Wins); default: throw new ArgumentException("Invalid Compare Type"); } } } c10.indd 239 30-01-2014 20:15:19 240  ❘  CHAPTER 10  Collections Note  The Compare method returns 0 if the two elements passed to it are equal with the order. If a value less than 0 is returned, the first argument is less than the second. With a value larger than 0, the first argument is greater than the second. Passing null with an argument, the method shouldn’t throw a NullReferenceException. Instead, null should take its place before any other element, thus –1 is returned if the first argu- ment is null, and +1 if the second argument is null. An instance of the RacerComparer class can now be used with the Sort method. Passing the enumeration RacerComparer.CompareType.Country sorts the collection by the property Country: racers.Sort(new RacerComparer(RacerComparer.CompareType.Country)); racers.ForEach(Console.WriteLine); Another way to do the sort is by using the overloaded Sort method, which requires a Comparison delegate: public void List.Sort(Comparison); Comparison is a delegate to a method that has two parameters of type T and a return type int. If the parameter values are equal, the method must return 0. If the first parameter is less than the second, a value less than zero must be returned; otherwise, a value greater than zero is returned: public delegate int Comparison(T x, T y); Now you can pass a lambda expression to the Sort method to do a sort by the number of wins. The two parameters are of type Racer, and in the implementation the Wins properties are compared by using the int method CompareTo. Also in the implementation, r2 and r1 are used in reverse order, so the number of wins is sorted in descending order. After the method has been invoked, the complete racer list is sorted based on the racer’s number of wins: racers.Sort((r1, r2) => r2.Wins.CompareTo(r1.Wins)); You can also reverse the order of a complete collection by invoking the Reverse method. Type Conversion With the List method ConvertAll, all types of a collection can be converted to a different type. The ConvertAll method uses a Converter delegate that is defined like this: public sealed delegate TOutput Converter(TInput from); The generic types TInput and TOutput are used with the conversion. TInput is the argument of the delegate method, and TOutput is the return type. In this example, all Racer types should be converted to Person types. Whereas the Racer type contains a firstName, lastName, country, and the number of wins, the Person type contains just a name. For the conversion, the country of the racer and the number of race wins can be ignored, but the name must be converted: [Serializable] public class Person { private string name; public Person(string name) { this.name = name; } public override string ToString() c10.indd 240 30-01-2014 20:15:19 Queues  ❘  241 { return name; } } The conversion happens by invoking the racers.ConvertAll method. The argument of this method is defined as a lambda expression with an argument of type Racer and a Person type that is returned. In the implementation of the lambda expression, a new Person object is created and returned. For the Person object, the FirstName and LastName are passed to the constructor: List persons = racers.ConvertAll( r => new Person(r.FirstName + " " + r.LastName)); The result of the conversion is a list containing the converted Person objects: persons of type List. Read-only Collections After collections are created they are read/write of course; otherwise, you couldn’t fill them with any values. However, after the collection is filled, you can create a read-only collection. The List col- lection has the method AsReadOnly that returns an object of type ReadOnlyCollection. The class ReadOnlyCollection implements the same interfaces as List, but all methods and properties that change the collection throw a NotSupportedException. Besides the interfaces of List, ReadOnlyCollection also implements the interfaces IReadOnlyCollection and IReadOnlyList. With the members of these interfaces, the collection cannot be changed. Queues A queue is a collection whose elements are processed first in, first out (FIFO), meaning the item that is put first in the queue is read first. Examples of queues are standing in line at the airport, a human resources queue to process employee applicants, print jobs waiting to be processed in a print queue, and a thread wait- ing for the CPU in a round-robin fashion. Sometimes the elements of a queue differ in their priority. For example, in the queue at the airport, business passengers are processed before economy passengers. In this case, multiple queues can be used, one queue for each priority. At the airport this is easily handled with sep- arate check-in queues for business and economy passengers. The same is true for print queues and threads. You can have an array or a list of queues whereby one item in the array stands for a priority. Within every array item there’s a queue, where processing happens using the FIFO principle. Note  Later in this chapter, a different implementation with a linked list is used to define a list of priorities. A queue is implemented with the Queue class in the namespace System.Collections.Generic. Internally, the Queue class is using an array of type T, similar to the List type. It implements the interfaces IEnumerable and ICollection; but not ICollection, which is not implemented because this interface defines Add and Remove methods that shouldn’t be available for queues. The Queue class does not implement the interface IList, so you cannot access the queue using an indexer. The queue just allows you to add an item to it, which is put at the end of the queue (with the Enqueue method), and to get items from the head of the queue (with the Dequeue method). Figure 10-1 shows the items of a queue. The Enqueue method adds items to one end of the queue; the items are read and removed at the other end of the queue with the Dequeue method. Invoking the Dequeue method once more removes the next item from the queue. c10.indd 241 30-01-2014 20:15:20 242  ❘  CHAPTER 10  Collections Methods of the Queue class are described in the following table. Selected Queue Members Description Count Returns the number of items in the queue. Enqueue Adds an item to the end of the queue. Dequeue Reads and removes an item from the head of the queue. If there are no more items in the queue when the Dequeue method is invoked, an excep- tion of type InvalidOperationException is thrown. Peek Reads an item from the head of the queue but does not remove the item. TrimExcess Resizes the capacity of the queue. The Dequeue method removes items from the queue, but it doesn’t resize the capacity of the queue. To get rid of the empty items at the beginning of the queue, use the TrimExcess method. When creating queues, you can use constructors similar to those used with the List type. The default constructor creates an empty queue, but you can also use a constructor to specify the capacity. As items are added to the queue, the capacity is increased to hold 4, 8, 16, and 32 items if the capacity is not defined. Similar to the List class, the capacity is always doubled as required. The default constructor of the non- generic Queue class is different, because it creates an initial array of 32 empty items. With an overload of the constructor, you can also pass any other collection that implements the IEnumerable interface that is copied to the queue. The following example demonstrating the use of the Queue class is a document management application. One thread is used to add documents to the queue, and another thread reads documents from the queue and processes them. The items stored in the queue are of type Document. The Document class defines a title and content (code file QueueSample/Document.cs): public class Document { public string Title { get; private set; } public string Content { get; private set; } public Document(string title, string content) { this.Title = title; this.Content = content; } } The DocumentManager class is a thin layer around the Queue class. It defines how to handle documents: adding documents to the queue with the AddDocument method, and getting documents from the queue with the GetDocument method. FIGURE 10-1 Enqueue Dequeue c10.indd 242 30-01-2014 20:15:21 Queues  ❘  243 Inside the AddDocument method, the document is added to the end of the queue using the Enqueue method. The first document from the queue is read with the Dequeue method inside GetDocument. Because mul- tiple threads can access the DocumentManager concurrently, access to the queue is locked with the lock statement. Note  Threading and the lock statement are discussed in Chapter 21, “Tasks, Threads, and Synchronization.” IsDocumentAvailable is a read-only Boolean property that returns true if there are documents in the queue, and false if not (code file QueueSample/DocumentManager.cs): public class DocumentManager { private readonly Queue documentQueue = new Queue(); public void AddDocument(Document doc) { lock (this) { documentQueue.Enqueue(doc); } } public Document GetDocument() { Document doc = null; lock (this) { doc = documentQueue.Dequeue(); } return doc; } public bool IsDocumentAvailable { get { return documentQueue.Count > 0; } } } The class ProcessDocuments processes documents from the queue in a separate task. The only method that can be accessed from the outside is Start. In the Start method, a new task is instantiated. A ProcessDocuments object is created to starting the task, and the Run method is defined as the start method of the task. The StartNew method of the TaskFactory (which is accessed from the static Factory property of the Task class) requires a delegate Action parameter where the address of the Run method can be passed to. The StartNew method of the TaskFactory immediately starts the task. With the Run method of the ProcessDocuments class, an endless loop is defined. Within this loop, the prop- erty IsDocumentAvailable is used to determine whether there is a document in the queue. If so, the docu- ment is taken from the DocumentManager and processed. Processing in this example is writing information c10.indd 243 30-01-2014 20:15:21 244  ❘  CHAPTER 10  Collections only to the console. In a real application, the document could be written to a file, written to the database, or sent across the network (code file QueueSample/ProcessDocuments.cs): public class ProcessDocuments { public static void Start(DocumentManager dm) { Task.Factory.StartNew(new ProcessDocuments(dm).Run); } protected ProcessDocuments(DocumentManager dm) { if (dm == null) throw new ArgumentNullException("dm"); documentManager = dm; } private DocumentManager documentManager; protected void Run() { while (true) { if (documentManager.IsDocumentAvailable) { Document doc = documentManager.GetDocument(); Console.WriteLine("Processing document {0}", doc.Title); } Thread.Sleep(new Random().Next(20)); } } } In the Main method of the application, a DocumentManager object is instantiated, and the document processing task is started. Then 1,000 documents are created and added to the DocumentManager (code file QueueSample/Program.cs): class Program { static void Main() { var dm = new DocumentManager(); ProcessDocuments.Start(dm); // Create documents and add them to the DocumentManager for (int i = 0; i < 1000; i++) { var doc = new Document("Doc " + i.ToString(), "content"); dm.AddDocument(doc); Console.WriteLine("Added document {0}", doc.Title); Thread.Sleep(new Random().Next(20)); } } } When you start the application, the documents are added to and removed from the queue, and you get out- put similar to the following: Added document Doc 279 Processing document Doc 236 Added document Doc 280 c10.indd 244 30-01-2014 20:15:21 Stacks  ❘  245 Processing document Doc 237 Added document Doc 281 Processing document Doc 238 Processing document Doc 239 Processing document Doc 240 Processing document Doc 241 Added document Doc 282 Processing document Doc 242 Added document Doc 283 Processing document Doc 243 A real-life scenario using the task described with the sample application might be an application that processes documents received with a web service. Stacks A stack is another container that is very similar to the queue. You just use different methods to access the stack. The item that is added last to the stack is read first, so the stack is a last in, first out (LIFO) container. Figure 10-2 shows the representation of a stack where the Push method adds an item to the stack, and the Pop method gets the item that was added last. Similar to the Queue class, the Stack class implements the interfaces IEnumerable and ICollection. Members of the Stack class are listed in the following table. Selected Stack Members Description Count Returns the number of items in the stack. Push Adds an item on top of the stack. Pop Removes and returns an item from the top of the stack. If the stack is empty, an exception of type InvalidOperationException is thrown. Peek Returns an item from the top of the stack but does not remove the item. Contains Checks whether an item is in the stack and returns true if it is. In this example, three items are added to the stack with the Push method. With the foreach method, all items are iterated using the IEnumerable interface. The enumerator of the stack does not remove the items; it just returns them item by item (code file StackSample/Program.cs): var alphabet = new Stack(); alphabet.Push('A'); alphabet.Push('B'); alphabet.Push('C'); foreach (char item in alphabet) { Console.Write(item); } Console.WriteLine(); FIGURE 10-2 Push Pop c10.indd 245 30-01-2014 20:15:23 246  ❘  CHAPTER 10  Collections Because the items are read in order from the last item added to the first, the following result is produced: CBA Reading the items with the enumerator does not change the state of the items. With the Pop method, every item that is read is also removed from the stack. This way, you can iterate the collection using a while loop and verify the Count property if items still exist: var alphabet = new Stack(); alphabet.Push('A'); alphabet.Push('B'); alphabet.Push('C'); Console.Write("First iteration: "); foreach (char item in alphabet) { Console.Write(item); } Console.WriteLine(); Console.Write("Second iteration: "); while (alphabet.Count > 0) { Console.Write(alphabet.Pop()); } Console.WriteLine(); The result gives CBA twice, once for each iteration. After the second iteration, the stack is empty because the second iteration used the Pop method: First iteration: CBA Second iteration: CBA Linked Lists LinkedList is a doubly linked list, whereby one element references the next and the previous one, as shown in Figure 10-3. This way you can easily walk through the complete list forward by moving to the next element, or backward by moving to the previous element. Value Next Previous Value Next Previous Value Next Previous Value Next Previous Figure 10-3 The advantage of a linked list is that if items are inserted anywhere in the list, the linked list is very fast. When an item is inserted, only the Next reference of the previous item and the Previous reference of the next item must be changed to reference the inserted item. With the List class, when an element is inserted all subsequent elements must be moved. Of course, there’s also a disadvantage with linked lists. Items of linked lists can be accessed only one after the other. It takes a long time to find an item that’s somewhere in the middle or at the end of the list. c10.indd 246 30-01-2014 20:15:24 Linked Lists  ❘  247 A linked list cannot just store the items inside the list; together with every item, the linked list must have information about the next and previous items. That’s why the LinkedList contains items of type LinkedListNode. With the class LinkedListNode, you can get to the next and previous items in the list. The LinkedListNode class defines the properties List, Next, Previous, and Value. The List property returns the LinkedList object that is associated with the node. Next and Previous are for iterating through the list and accessing the next or previous item. Value returns the item that is associated with the node. Value is of type T. The LinkedList class itself defines members to access the first (First) and last (Last) item of the list, to insert items at specific positions (AddAfter, AddBefore, AddFirst, AddLast), to remove items from specific positions (Remove, RemoveFirst, RemoveLast), and to find elements where the search starts from either the beginning (Find) or the end (FindLast) of the list. The sample application to demonstrate linked lists uses a linked list together with a list. The linked list contains documents as in the queue example, but the documents have an additional priority associated with them. The documents will be sorted inside the linked list depending on the priority. If multiple documents have the same priority, the elements are sorted according to the time when the document was inserted. Figure 10-4 describes the collections of the sample application. LinkedList is the linked list containing all the Document objects. The figure shows the title and priority of the documents. The title indicates when the document was added to the list: The first document added has the title "One", the second document has the title "Two", and so on. You can see that the documents One and Four have the same prior- ity, 8, but because One was added before Four, it is earlier in the list. When new documents are added to the linked list, they should be added after the last docu- ment that has the same priority. The LinkedList collection contains elements of type LinkedListNode. The class LinkedListNode adds Next and Previous properties to walk from one node to the next. For referencing such elements, the List is defined as List>. For fast access to the last document of every priority, the collection List contains up to 10 elements, each referencing the last document of every priority. In the upcoming discussion, the reference to the last document of every priority is called the priority node. Using the previous example, the Document class is extended to contain the priority, which is set with the constructor of the class (code file LinkedListSample/Document.cs): public class Document { public string Title { get; private set; } public string Content { get; private set; } public byte Priority { get; private set; } public Document(string title, string content, byte priority) { this.Title = title; this.Content = content; this.Priority = priority; } } The heart of the solution is the PriorityDocumentManager class. This class is very easy to use. With the public interface of this class, new Document elements can be added to the linked list, the first document can be retrieved, and for testing purposes it also has a method to display all elements of the collection as they are linked in the list. The class PriorityDocumentManager contains two collections. The collection of type LinkedList contains all documents. The collection of type List> contains references of up to 10 elements that are entry points for adding new documents with a specific priority. c10.indd 247 30-01-2014 20:15:25 248  ❘  CHAPTER 10  Collections Both collection variables are initialized with the constructor of the class PriorityDocumentManager. The list collection is also initialized with null (code file LinkedListSample/PriorityDocumentManager.cs): public class PriorityDocumentManager { private readonly LinkedList documentList; // priorities 0.9 private readonly List> priorityNodes; public PriorityDocumentManager() { documentList = new LinkedList(); priorityNodes = new List>(10); for (int i = 0; i < 10; i++) { priorityNodes.Add(new LinkedListNode(null)); } } Figure 10-4 LinkedList Six 9 9 8 7 6 5 4 3 2 1 0 One 8 Four 8 Three 4 Two 3 Five 1 Seven 1 Eight 1 List> c10.indd 248 30-01-2014 20:15:26 Linked Lists  ❘  249 Part of the public interface of the class is the method AddDocument. AddDocument does nothing more than call the private method AddDocumentToPriorityNode. The reason for having the implementation inside a different method is that AddDocumentToPriorityNode may be called recursively, as you will see soon: public void AddDocument(Document d) { if (d == null) throw new ArgumentNullException("d"); AddDocumentToPriorityNode(d, d.Priority); } The first action that is done in the implementation of AddDocumentToPriorityNode is a check to see if the priority fits in the allowed priority range. Here, the allowed range is between 0 and 9. If a wrong value is passed, an exception of type ArgumentException is thrown. Next, you check whether there’s already a priority node with the same priority as the priority that was passed. If there’s no such priority node in the list collection, AddDocumentToPriorityNode is invoked recursively with the priority value decremented to check for a priority node with the next lower priority. If there’s no priority node with the same priority or any priority with a lower value, the document can be safely added to the end of the linked list by calling the method AddLast. In addition, the linked list node is referenced by the priority node that’s responsible for the priority of the document. If there’s an existing priority node, you can get the position inside the linked list where the document should be inserted. In the following example, you must determine whether a priority node already exists with the correct priority, or if there’s just a priority node that references a document with a lower priority. In the first case, you can insert the new document after the position referenced by the priority node. Because the prior- ity node always must reference the last document with a specific priority, the reference of the priority node must be set. It gets more complex if only a priority node referencing a document with a lower priority exists. Here, the document must be inserted before all documents with the same priority as the priority node. To get the first document of the same priority, a while loop iterates through all linked list nodes, using the Previous property, until a linked list node is reached that has a different priority. This way, you know the position where the document must be inserted, and the priority node can be set: private void AddDocumentToPriorityNode(Document doc, int priority) { if (priority > 9 || priority < 0) throw new ArgumentException("Priority must be between 0 and 9"); if (priorityNodes[priority].Value == null) { ––priority; if (priority >= 0) { // check for the next lower priority AddDocumentToPriorityNode(doc, priority); } else // now no priority node exists with the same priority or lower // add the new document to the end { documentList.AddLast(doc); priorityNodes[doc.Priority] = documentList.Last; } return; } else // a priority node exists { LinkedListNode prioNode = priorityNodes[priority]; if (priority == doc.Priority) // priority node with the same priority exists { c10.indd 249 30-01-2014 20:15:26 250  ❘  CHAPTER 10  Collections documentList.AddAfter(prioNode, doc); // set the priority node to the last document with the same priority priorityNodes[doc.Priority] = prioNode.Next; } else // only priority node with a lower priority exists { // get the first node of the lower priority LinkedListNode firstPrioNode = prioNode; while (firstPrioNode.Previous != null && firstPrioNode.Previous.Value.Priority == prioNode.Value.Priority) { firstPrioNode = prioNode.Previous; prioNode = firstPrioNode; } documentList.AddBefore(firstPrioNode, doc); // set the priority node to the new value priorityNodes[doc.Priority] = firstPrioNode.Previous; } } } Now only simple methods are left for discussion. DisplayAllNodes does a foreach loop to display the pri- ority and the title of every document to the console. The method GetDocument returns the first document (the document with the highest priority) from the linked list and removes it from the list: public void DisplayAllNodes() { foreach (Document doc in documentList) { Console.WriteLine("priority: {0}, title {1}", doc.Priority, doc.Title); } } // returns the document with the highest priority // (that's first in the linked list) public Document GetDocument() { Document doc = documentList.First.Value; documentList.RemoveFirst(); return doc; } } In the Main method, the PriorityDocumentManager is used to demonstrate its functionality. Eight new documents with different priorities are added to the linked list, and then the complete list is displayed (code file LinkedListSample/Program.cs): static void Main() { var pdm = new PriorityDocumentManager(); pdm.AddDocument(new Document("one", "Sample", 8)); pdm.AddDocument(new Document("two", "Sample", 3)); pdm.AddDocument(new Document("three", "Sample", 4)); pdm.AddDocument(new Document("four", "Sample", 8)); pdm.AddDocument(new Document("five", "Sample", 1)); pdm.AddDocument(new Document("six", "Sample", 9)); c10.indd 250 30-01-2014 20:15:26 Sorted List  ❘  251 pdm.AddDocument(new Document("seven", "Sample", 1)); pdm.AddDocument(new Document("eight", "Sample", 1)); pdm.DisplayAllNodes(); } With the processed result, you can see that the documents are sorted first by priority and second by when the document was added: priority: 9, title six priority: 8, title one priority: 8, title four priority: 4, title three priority: 3, title two priority: 1, title five priority: 1, title seven priority: 1, title eight Sorted List If the collection you need should be sorted based on a key, you can use the SortedList. This class sorts the elements based on a key. You can use any type for the value, and also for the key. The following example creates a sorted list for which both the key and the value are of type string. The default constructor creates an empty list, and then two books are added with the Add method. With over- loaded constructors, you can define the capacity of the list and pass an object that implements the interface IComparer, which is used to sort the elements in the list. The first parameter of the Add method is the key (the book title); the second parameter is the value (the ISBN number). Instead of using the Add method, you can use the indexer to add elements to the list. The indexer requires the key as an index parameter. If a key already exists, the Add method throws an exception of type ArgumentException. If the same key is used with the indexer, the new value replaces the old value (code file SortedListSample/Program.cs): var books = new SortedList(); books.Add("Professional WPF Programming", "978–0–470–04180–2"); books.Add("Professional ASP.NET MVC 3", "978–1–1180–7658–3"); books["Beginning Visual C# 2010"] = "978–0–470-50226-6"; books["Professional C# 4 and .NET 4"] = "978–0–470–50225–9"; Note  SortedList allows only one value per key. If you need mul- tiple values per key you can use Lookup. You can iterate through the list using a foreach statement. Elements returned by the enumerator are of type KeyValuePair, which contains both the key and the value. The key can be accessed with the Key property, and the value can be accessed with the Value property: foreach (KeyValuePair book in books) { Console.WriteLine("{0}, {1}", book.Key, book.Value); } The iteration displays book titles and ISBN numbers ordered by the key: Beginning Visual C# 2010, 978-0-470-50226-6 Professional ASP.NET MVC 3, 978-1-1180-7658-3 Professional C# 4 and .NET 4, 978-0-470-50225-9 Professional WPF Programming, 978-0-470-04180-2 c10.indd 251 30-01-2014 20:15:27 252  ❘  CHAPTER 10  Collections You can also access the values and keys by using the Values and Keys properties. The Values property returns IList and the Keys property returns IList, so you can use these properties with a foreach statement: foreach (string isbn in books.Values) { Console.WriteLine(isbn); } foreach (string title in books.Keys) { Console.WriteLine(title); } The first loop displays the values, and next the keys: 978-0-470-50226-6 978-1-1180-7658-3 978-0-470-50225-9 978-0-470-04180-2 Beginning Visual C# 2010 Professional ASP.NET MVC 3 Professional C# 4 and .NET 4 Professional WPF Programming If you try to access an element with an indexer and passing a key that does not exist, an exception of type KeyNotFoundException is thrown. To avoid that exception you can use the method ContainsKey, which returns true if the key passed exists in the collection, or you can invoke the method TryGetValue, which tries to get the value but doesn’t throw an exception if it isn’t found: string isbn; string title = "Professional C# 7.0"; if (!books.TryGetValue(title, out isbn)) { Console.WriteLine("{0} not found", title); } Dictionaries A dictionary represents a sophisticated data structure that enables you to access an element based on a key. Dictionaries are also known as hash tables or maps. The main feature of dictionaries is fast lookup based on keys. You can also add and remove items freely, a bit like a List, but without the performance overhead of having to shift subsequent items in memory. Figure 10-5 shows a simplified representation of a dictionary. Here employee-ids such as B4711 are the keys added to the dictionary. The key is transformed into a hash. With the hash a number is created to associate an index with the values. The index then contains a link to the value. The figure is simplified because it is possible for a single index entry to be associated with multiple values, and the index can be stored as a tree. The .NET Framework offers several dictionary classes. The main class to use is Dictionary. Key Type A type that is used as a key in the dictionary must override the method GetHashCode of the Object class. Whenever a dictionary class needs to determine where an item should be located, it calls the GetHashCode method. The int that is returned by GetHashCode is used by the dictionary to calculate an index of where to place the element. We won’t go into this part of the algorithm; what you should know is that it involves prime numbers, so the capacity of a dictionary is a prime number. c10.indd 252 30-01-2014 20:15:27 Dictionaries  ❘  253 The implementation of GetHashCode must satisfy the following requirements: ➤➤ The same object should always return the same value. ➤➤ Different objects can return the same value. ➤➤ It should execute as quickly as possible; it must be inexpensive to compute. ➤➤ It must not throw exceptions. ➤➤ It should use at least one instance field. ➤➤ The hash code value should be evenly distributed across the entire range of numbers that an int can store. ➤➤ The hash code should not change during the lifetime of the object. Note  Good performance of the dictionary is based on a good implementation of the method GetHashCode. What’s the reason for having hash code values evenly distributed across the range of integers? If two keys return hashes that have the same index, the dictionary class needs to start looking for the nearest available free location to store the second item — and it will have to do some searching to retrieve this item later. This is obviously going to hurt performance. In addition, if a lot of your keys are tending to provide the same storage indexes for where they should be stored, this kind of clash becomes more likely. However, because of the way that Microsoft’s part of the algorithm works, this risk is minimized when the calculated hash values are evenly distributed between int.MinValue and int.MaxValue. Besides having an implementation of GetHashCode, the key type also must implement the IEquatable .Equals method or override the Equals method from the Object class. Because different key objects may return the same hash code, the method Equals is used by the dictionary comparing keys. The dictionary 61 Tony Stewart Index ValuesKeys B12836 B4711 B12836 N34434 0 1 2 31 32 60 Jimmie JohnsonB4711 Matt KensethN34434 . . . . . . Figure 10-5 c10.indd 253 30-01-2014 20:15:28 254  ❘  CHAPTER 10  Collections examines whether two keys, such as A and B, are equal, invoking A.Equals(B). This means that you must ensure that the following is always true: If A.Equals(B) is true, then A.GetHashCode and B.GetHashCode must always return the same hash code. This may seem a fairly subtle point, but it is crucial. If you contrived some way of overriding these methods so that the preceding statement were not always true, a dictionary that uses instances of this class as its keys would not work properly. Instead, you’d find funny things happening. For example, you might place an object in the dictionary and then discover that you could never retrieve it, or you might try to retrieve an entry and have the wrong entry returned. Note  For this reason, the C# compiler displays a compilation warning if you supply an override for Equals but don’t supply an override for GetHashCode. For System.Object this condition is true because Equals simply compares references, and GetHashCode actually returns a hash that is based solely on the address of the object. This means that hash tables based on a key that doesn’t override these methods will work correctly. However, the problem with this approach is that keys are regarded as equal only if they are the same object. That means when you place an object in the dictionary, you have to hang onto the reference to the key; you can’t simply instantiate another key object later with the same value. If you don’t override Equals and GetHashCode, the type is not very conve- nient to use in a dictionary. Incidentally, System.String implements the interface IEquatable and overloads GetHashCode appropri- ately. Equals provides value comparison, and GetHashCode returns a hash based on the value of the string. Strings can be used conveniently as keys in dictionaries. Number types such as Int32 also implement the interface IEquatable and overload GetHashCode. However, the hash code returned by these types simply maps to the value. If the number you would like to use as a key is not itself distributed around the possible values of an integer, using integers as keys doesn’t fulfill the rule of evenly distributing key values to get the best performance. Int32 is not meant to be used in a dictionary. If you need to use a key type that does not implement IEquatable and override GetHashCode accord- ing to the key values you store in the dictionary, you can create a comparer implementing the interface IEqualityComparer. IEqualityComparer defines the methods GetHashCode and Equals with an argument of the object passed, so you can offer an implementation different from the object type itself. An overload of the Dictionary constructor allows passing an object implementing IEqualityComparer. If such an object is assigned to the dictionary, this class is used to generate the hash codes and compare the keys. Dictionary Example The dictionary example in this section is a program that sets up a dictionary of employees. The dictionary is indexed by EmployeeId objects, and each item stored in the dictionary is an Employee object that stores details of an employee. The struct EmployeeId is implemented to define a key to be used in a dictionary. The members of the class are a prefix character and a number for the employee. Both of these variables are read-only and can be initialized only in the constructor to ensure that keys within the dictionary shouldn’t change, and this way that is guaranteed. The fields are filled within the constructor. The ToString method is overloaded to get a string representation of the employee ID. As required for a key type, EmployeeId implements the interface IEquatable and overloads the method GetHashCode (code file DictionarySample/EmployeeId.cs): c10.indd 254 30-01-2014 20:15:29 Dictionaries  ❘  255 [Serializable] public class EmployeeIdException : Exception { public EmployeeIdException(string message) : base(message) { } } [Serializable] public struct EmployeeId : IEquatable { private readonly char prefix; private readonly int number; public EmployeeId(string id) { Contract.Requires(id != null); prefix = (id.ToUpper())[0]; int numLength = id.Length - 1; try { number = int.Parse(id.Substring(1, numLength > 6 ? 6 : numLength)); } catch (FormatException) { throw new EmployeeIdException("Invalid EmployeeId format"); } } public override string ToString() { return prefix.ToString() + string.Format("{0,6:000000}", number); } public override int GetHashCode() { return (number ^ number << 16) * 0x15051505; } public bool Equals(EmployeeId other) { if (other == null) return false; return (prefix == other.prefix && number == other.number); } public override bool Equals(object obj) { return Equals((EmployeeId)obj); } public static bool operator ==(EmployeeId left, EmployeeId right) { return left.Equals(right); } public static bool operator !=(EmployeeId left, EmployeeId right) { return !(left == right); } } The Equals method that is defined by the IEquatable interface compares the values of two EmployeeId objects and returns true if both values are the same. Instead of implementing the Equals c10.indd 255 30-01-2014 20:15:29 256  ❘  CHAPTER 10  Collections method from the IEquatable interface, you can also override the Equals method from the Object class: public bool Equals(EmployeeId other) { if (other == null) return false; return (prefix == other.prefix && number == other.number); } With the number variable, a value from 1 to around 190,000 is expected for the employees. This doesn’t fill the range of an integer. The algorithm used by GetHashCode shifts the number 16 bits to the left, then does an XOR with the original number, and finally multiplies the result by the hex value 15051505. The hash code is fairly evenly distributed across the range of an integer: public override int GetHashCode() { return (number ^ number << 16) * 0x15051505; } Note  On the Internet, you can find a lot more complex algorithms that have a better distribution across the integer range. You can also use the GetHashCode method of a string to return a hash. The Employee class is a simple entity class containing the name, salary, and ID of the employee. The con- structor initializes all values, and the method ToString returns a string representation of an instance. The implementation of ToString uses a format string to create the string representation for performance reasons (code file DictionarySample/Employee.cs): [Serializable] public class Employee { private string name; private decimal salary; private readonly EmployeeId id; public Employee(EmployeeId id, string name, decimal salary) { this.id = id; this.name = name; this.salary = salary; } public override string ToString() { return String.Format("{0}: {1, -20} {2:C}", id.ToString(), name, salary); } } In the Main method of the sample application, a new Dictionary instance is created, where the key is of type EmployeeId and the value is of type Employee. The constructor allocates a capac- ity of 31 elements. Remember that capacity is based on prime numbers. However, when you assign a value that is not a prime number, you don’t need to worry. The Dictionary class itself takes the next prime number that follows the integer passed to the constructor to allocate the capacity. The employee objects and IDs are created and added to the dictionary with the Add method. Instead of using the Add method, you can also use the indexer to add keys and values to the dictionary, as shown here with the employees Matt and Brad (code file DictionarySample/Program.cs): c10.indd 256 30-01-2014 20:15:29 Dictionaries  ❘  257 static void Main() { var employees = new Dictionary(31); var idTony = new EmployeeId("C3755"); var tony = new Employee(idTony, "Tony Stewart", 379025.00m); employees.Add(idTony, tony); Console.WriteLine(tony); var idCarl = new EmployeeId("F3547"); var carl = new Employee(idCarl, "Carl Edwards", 403466.00m); employees.Add(idCarl, carl); Console.WriteLine(carl); var idKevin = new EmployeeId("C3386"); var kevin = new Employee(idKevin, "Kevin Harwick", 415261.00m); employees.Add(idKevin, kevin); Console.WriteLine(kevin); var idMatt = new EmployeeId("F3323"); var matt = new Employee(idMatt, "Matt Kenseth", 1589390.00m); employees[idMatt] = matt; Console.WriteLine(matt); var idBrad = new EmployeeId("D3234"); var brad = new Employee(idBrad, "Brad Keselowski", 322295.00m); employees[idBrad] = brad; Console.WriteLine(brad); After the entries are added to the dictionary, inside a while loop employees are read from the dictionary. The user is asked to enter an employee number to store in the variable userInput, and the user can exit the application by entering X. If the key is in the dictionary, it is examined with the TryGetValue method of the Dictionary class. TryGetValue returns true if the key is found and false otherwise. If the value is found, the value associated with the key is stored in the employee variable. This value is written to the console. Note  You can also use an indexer of the Dictionary class instead of TryGetValue to access a value stored in the dictionary. However, if the key is not found, the indexer throws an exception of type KeyNotFoundException. while (true) { Console.Write("Enter employee id (X to exit)> "); var userInput = Console.ReadLine(); userInput = userInput.ToUpper(); if (userInput == "X") break; EmployeeId id; try { id = new EmployeeId(userInput); Employee employee; if (!employees.TryGetValue(id, out employee)) { Console.WriteLine("Employee with id {0} does not exist", id); } else { Console.WriteLine(employee); } } catch (EmployeeIdException ex) c10.indd 257 30-01-2014 20:15:29 258  ❘  CHAPTER 10  Collections { Console.WriteLine(ex.Message); } } } } Running the application produces the following output: Enter employee id (X to exit)> C3386 C003386: Kevin Harwick $415,261.00 Enter employee id (X to exit)> F3547 F003547: Carl Edwards $403,466.00 Enter employee id (X to exit)> X Press any key to continue ... Lookups Dictionary supports only one value per key. The class Lookup resem- bles a Dictionary but maps keys to a collection of values. This class is implemented in the assembly System.Core and defined with the namespace System.Linq. Lookup cannot be created as a normal dictionary. Instead, you have to invoke the method ToLookup, which returns a Lookup object. The method ToLookup is an exten- sion method that is available with every class implementing IEnumerable. In the following example, a list of Racer objects is filled. Because List implements IEnumerable, the ToLookup method can be invoked on the racers list. This method requires a delegate of type Func that defines the selector of the key. Here, the racers are selected based on their country by using the lambda expression r => r.Country. The foreach loop accesses only the racers from Australia by using the indexer (code file LookupSample/Program.cs): var racers = new List(); racers.Add(new Racer("Jacques", "Villeneuve", "Canada", 11)); racers.Add(new Racer("Alan", "Jones", "Australia", 12)); racers.Add(new Racer("Jackie", "Stewart", "United Kingdom", 27)); racers.Add(new Racer("James", "Hunt", "United Kingdom", 10)); racers.Add(new Racer("Jack", "Brabham", "Australia", 14)); var lookupRacers = racers.ToLookup(r => r.Country); foreach (Racer r in lookupRacers["Australia"]) { Console.WriteLine(r); } Note  You can read more about extension methods in Chapter 11, “Language Integrated Query.” Lambda expressions are explained in Chapter 8. The output shows the racers from Australia: Alan Jones Jack Brabham Sorted Dictionaries SortedDictionary is a binary search tree in which the items are sorted based on the key. The key type must implement the interface IComparable. If the key type is not sortable, you can also create a comparer implementing IComparer and assign the comparer as a constructor argument of the sorted dictionary. c10.indd 258 30-01-2014 20:15:29 Sets  ❘  259 Earlier in this chapter you read about SortedList. SortedDictionary and SortedList have similar functionality, but because SortedList is implemented as a list that is based on an array, and SortedDictionary is implemented as a dictionary, the classes have different characteristics: ➤➤ SortedList uses less memory than SortedDictionary. ➤➤ SortedDictionary has faster insertion and removal of elements. ➤➤ When populating the collection with already sorted data, SortedList is faster if capacity changes are not needed. Note  SortedList consumes less memory than SortedDictionary. SortedDictionary is faster with inserts and the removal of unsorted data. Sets A collection that contains only distinct items is known by the term set. The .NET Framework includes two sets, HashSet and SortedSet, that both implement the interface ISet. HashSet contains an unordered list of distinct items; with SortedSet the list is ordered. The ISet interface offers methods to create a union of multiple sets, an intersection of sets, or to provide information if one set is a superset or subset of another. In the following sample code, three new sets of type string are created and filled with Formula-1 cars. The HashSet class implements the ICollection interface. However, the Add method is implemented explicitly and a different Add method is offered by the class, as you can see here. The Add method differs by the return type; a Boolean value is returned to provide the information if the element was added. If the element was already in the set, it is not added, and false is returned (code file SetSample/Program.cs): var companyTeams = new HashSet() { "Ferrari", "McLaren", "Mercedes" }; var traditionalTeams = new HashSet() { "Ferrari", "McLaren" }; var privateTeams = new HashSet() { "Red Bull", "Toro Rosso", "Force India", "Sauber" }; if (privateTeams.Add("Williams")) Console.WriteLine("Williams added"); if (!companyTeams.Add("McLaren")) Console.WriteLine("McLaren was already in this set"); The result of these two Add methods is written to the console: Williams added McLaren was already in this set The methods IsSubsetOf and IsSupersetOf compare a set with a collection that implements the IEnumerable interface and returns a Boolean result. Here, IsSubsetOf verifies whether every element in traditionalTeams is contained in companyTeams, which is the case; IsSupersetOf verifies whether traditionalTeams has any additional elements compared to companyTeams: if (traditionalTeams.IsSubsetOf(companyTeams)) { Console.WriteLine("traditionalTeams is subset of companyTeams"); } if (companyTeams.IsSupersetOf(traditionalTeams)) { Console.WriteLine("companyTeams is a superset of traditionalTeams"); } c10.indd 259 30-01-2014 20:15:29 260  ❘  CHAPTER 10  Collections The output of this verification is shown here: traditionalTeams is a subset of companyTeams companyTeams is a superset of traditionalTeams Williams is a traditional team as well, which is why this team is added to the traditionalTeams collection: traditionalTeams.Add("Williams"); if (privateTeams.Overlaps(traditionalTeams)) { Console.WriteLine("At least one team is the same with the " + "traditional and private teams"); } Because there’s an overlap, this is the result: At least one team is the same with the traditional and private teams. The variable allTeams that references a new SortedSet is filled with a union of companyTeams, privateTeams, and traditionalTeams by calling the UnionWith method: var allTeams = new SortedSet(companyTeams); allTeams.UnionWith(privateTeams); allTeams.UnionWith(traditionalTeams); Console.WriteLine(); Console.WriteLine("all teams"); foreach (var team in allTeams) { Console.WriteLine(team); } Here, all teams are returned but every team is listed just once because the set contains only unique values; and because the container is a SortedSet, the result is ordered: Ferrari Force India Lotus McLaren Mercedes Red Bull Sauber Toro Rosso Williams The method ExceptWith removes all private teams from the allTeams set: allTeams.ExceptWith(privateTeams); Console.WriteLine(); Console.WriteLine("no private team left"); foreach (var team in allTeams) { Console.WriteLine(team); } The remaining elements in the collection do not contain any private team: Ferrari McLaren Mercedes Observable Collections In case you need information when items in the collection are removed or added, you can use the ObservableCollection class. This class was defined for WPF so that the UI is informed about collec- tion changes; therefore, this class is defined in the assembly WindowsBase and you need to reference it. The namespace of this class is System.Collections.ObjectModel. c10.indd 260 30-01-2014 20:15:30 Observable Collections  ❘  261 ObservableCollection derives from the base class Collection that can be used to create custom collections and it uses List internal. From the base class, the virtual methods SetItem and RemoveItem are overridden to fire the CollectionChanged event. Clients of this class can register to this event by using the interface INotifyCollectionChanged. The next example demonstrates using an ObservableCollection where the method Data_CollectionChanged is registered to the CollectionChanged event. Two items are added to the end — one item is inserted, and one item is removed (code file ObservableCollectionSample/Program.cs): var data = new ObservableCollection(); data.CollectionChanged += Data_CollectionChanged; data.Add("One"); data.Add("Two"); data.Insert(1, "Three"); data.Remove("One"); The method Data_CollectionChanged receives NotifyCollectionChangedEventArgs containing infor- mation about changes to the collection. The Action property provides information if an item was added or removed. With removed items, the OldItems property is set and lists the removed items. With added items, the NewItems property is set and lists the new items: static void Data_CollectionChanged(object sender, NotifyCollectionChangedEventArgs e) { Console.WriteLine("action: {0}", e.Action.ToString()); if (e.OldItems != null) { Console.WriteLine("starting index for old item(s): {0}", e.OldStartingIndex); Console.WriteLine("old item(s):"); foreach (var item in e.OldItems) { Console.WriteLine(item); } } if (e.NewItems != null) { Console.WriteLine("starting index for new item(s): {0}", e.NewStartingIndex); Console.WriteLine("new item(s): "); foreach (var item in e.NewItems) { Console.WriteLine(item); } } Console.WriteLine(); } Running the application results in the following output. First the items One and Two are added to the collec- tion, and thus the Add action is shown with the indexes 0 and 1. The third item, Three, is inserted on posi- tion 1 so it shows the action Add with index 1. Finally, the item One is removed as shown with the action Remove and index 0: action: Add starting index for new item(s): 0 new item(s): One action: Add starting index for new item(s): 1 new item(s): c10.indd 261 30-01-2014 20:15:30 262  ❘  CHAPTER 10  Collections Two action: Add starting index for new item(s): 1 new item(s): Three action: Remove starting index for old item(s): 0 old item(s): One Bit Arrays If you need to deal with a number of bits, you can use the class BitArray and the struct BitVector32. BitArray is located in the namespace System.Collections; BitVector32 is in the namespace System .Collections.Specialized. The most important difference between these two types is that BitArray is resizable, which is useful if you don’t know the number of bits needed in advance, and it can contain a large number of bits. BitVector32 is stack-based and therefore faster. BitVector32 contains only 32 bits, which are stored in an integer. BitArray The class BitArray is a reference type that contains an array of ints, where for every 32 bits a new integer is used. Members of this class are described in the following table. BitArray Members Description Count Length The get accessor of both Count and Length return the number of bits in the array. With the Length property, you can also define a new size and resize the collection. Item Get Set You can use an indexer to read and write bits in the array. The indexer is of type bool. Instead of using the indexer, you can also use the Get and Set methods to access the bits in the array. SetAll The method SetAll sets the values of all bits according to the parameter passed to the method. Not The method Not generates the inverse of all bits of the array. And Or Xor With the methods And, Or, and Xor, you can combine two BitArray objects. The And method does a binary AND, where the result bits are set only if the bits from both input arrays are set. The Or method does a binary OR, where the result bits are set if one or both of the input arrays are set. The Xor method is an exclusive OR, where the result is set if only one of the input bits is set. The helper method DisplayBits iterates through a BitArray and displays 1 or 0 to the console, depending on whether or not the bit is set (code file BitArraySample/Program.cs): static void DisplayBits(BitArray bits) { foreach (bool bit in bits) { Console.Write(bit ? 1: 0); } } c10.indd 262 30-01-2014 20:15:30 Bit Arrays  ❘  263 The example to demonstrate the BitArray class creates a bit array with 8 bits, indexed from 0 to 7. The SetAll method sets all 8 bits to true. Then the Set method changes bit 1 to false. Instead of the Set method, you can also use an indexer, as shown with indexes 5 and 7: var bits1 = new BitArray(8); bits1.SetAll(true); bits1.Set(1, false); bits1[5] = false; bits1[7] = false; Console.Write("initialized: "); DisplayBits(bits1); Console.WriteLine(); This is the displayed result of the initialized bits: initialized: 10111010 The Not method generates the inverse of the bits of the BitArray: Console.Write(" not "); DisplayBits(bits1); bits1.Not(); Console.Write(" = "); DisplayBits(bits1); Console.WriteLine(); The result of Not is all bits inversed. If the bit were true, it is false; and if it were false, it is true: not 10111010 = 01000101 In the following example, a new BitArray is created. With the constructor, the variable bits1 is used to initialize the array, so the new array has the same values. Then the values for bits 0, 1, and 4 are set to dif- ferent values. Before the Or method is used, the bit arrays bits1 and bits2 are displayed. The Or method changes the values of bits1: var bits2 = new BitArray(bits1); bits2[0] = true; bits2[1] = false; bits2[4] = true; DisplayBits(bits1); Console.Write(" or "); DisplayBits(bits2); Console.Write(" = "); bits1.Or(bits2); DisplayBits(bits1); Console.WriteLine(); With the Or method, the set bits are taken from both input arrays. In the result, the bit is set if it was set with either the first or the second array: 01000101 or 10001101 = 11001101 Next, the And method is used to operate on bits2 and bits1: DisplayBits(bits2); Console.Write(" and "); DisplayBits(bits1); Console.Write(" = "); bits2.And(bits1); DisplayBits(bits2); Console.WriteLine(); c10.indd 263 30-01-2014 20:15:30 264  ❘  CHAPTER 10  Collections The result of the And method only sets the bits where the bit was set in both input arrays: 10001101 and 11001101 = 10001101 Finally, the Xor method is used for an exclusive OR: DisplayBits(bits1); Console.Write(" xor "); DisplayBits(bits2); bits1.Xor(bits2); Console.Write(" = "); DisplayBits(bits1); Console.WriteLine(); With the Xor method, the resultant bits are set only if the bit was set either in the first or the second input, but not both: 11001101 xor 10001101 = 01000000 BitVector32 If you know the number of bits you need in advance, you can use the BitVector32 structure instead of BitArray. BitVector32 is more efficient because it is a value type and stores the bits on the stack inside an integer. With a single integer you have a place for 32 bits. If you need more bits, you can use multiple BitVector32 values or the BitArray. The BitArray can grow as needed; this is not an option with BitVector32. The following table shows the members of BitVector that are very different from BitArray. BitVector Members Description Data The property Data returns the data behind BitVector32 as an integer. Item The values for BitVector32 can be set using an indexer. The indexer is overloaded — you can get and set the values using a mask or a section of type BitVector32.Section. CreateMask CreateMask is a static method that you can use to create a mask for accessing specific bits in the BitVector32. CreateSection CreateSection is a static method that you can use to create several sections within the 32 bits. The following example creates BitVector32 with the default constructor, whereby all 32 bits are initialized to false. Then masks are created to access the bits inside the bit vector. The first call to CreateMask creates a mask to access the first bit. After CreateMask is invoked, bit1 has a value of 1. Invoking CreateMask once more and passing the first mask as a parameter to CreateMask returns a mask to access the second bit, which is 2. bit3 then has a value of 4 to access bit number 3, and bit4 has a value of 8 to access bit number 4. Then the masks are used with the indexer to access the bits inside the bit vector and to set the fields accord- ingly (code file BitArraySample/Program.cs): var bits1 = new BitVector32(); int bit1 = BitVector32.CreateMask(); int bit2 = BitVector32.CreateMask(bit1); int bit3 = BitVector32.CreateMask(bit2); int bit4 = BitVector32.CreateMask(bit3); int bit5 = BitVector32.CreateMask(bit4); bits1[bit1] = true; c10.indd 264 30-01-2014 20:15:30 Bit Arrays  ❘  265 bits1[bit2] = false; bits1[bit3] = true; bits1[bit4] = true; bits1[bit5] = true; Console.WriteLine(bits1); BitVector32 has an overridden ToString method that not only displays the name of the class but also 1 or 0 if the bits are set or not, respectively: BitVector32{00000000000000000000000000011101} Instead of creating a mask with the CreateMask method, you can define the mask yourself; you can also set multiple bits at once. The hexadecimal value abcdef is the same as the binary value 1010 1011 1100 1101 1110 1111. All the bits defined with this value are set: bits1[0xabcdef] = true; Console.WriteLine(bits1); With the output shown you can verify the bits that are set: BitVector32{00000000101010111100110111101111} Separating the 32 bits to different sections can be extremely useful. For example, an IPv4 address is defined as a four-byte number that is stored inside an integer. You can split the integer by defining four sections. With a multicast IP message, several 32-bit values are used. One of these 32-bit values is separated in these sections: 16 bits for the number of sources, 8 bits for a querier’s query interval code, 3 bits for a querier’s robustness variable, a 1-bit suppress flag, and 4 bits that are reserved. You can also define your own bit meanings to save memory. The following example simulates receiving the value 0x79abcdef and passes this value to the constructor of BitVector32, so that the bits are set accordingly: int received = 0x79abcdef; BitVector32 bits2 = new BitVector32(received); Console.WriteLine(bits2); The bits are shown on the console as initialized: BitVector32{01111001101010111100110111101111} Then six sections are created. The first section requires 12 bits, as defined by the hexadecimal value 0xfff (12 bits are set); section B requires 8 bits; section C, 4 bits; section D and E, 3 bits; and section F, 2 bits. The first call to CreateSection just receives 0xfff to allocate the first 12 bits. With the second call to CreateSection, the first section is passed as an argument, so the next section continues where the first sec- tion ended. CreateSection returns a value of type BitVector32.Section that contains the offset and the mask for the section: // sections: FF EEE DDD CCCC BBBBBBBB // AAAAAAAAAAAA BitVector32.Section sectionA = BitVector32.CreateSection(0xfff); BitVector32.Section sectionB = BitVector32.CreateSection(0xff, sectionA); BitVector32.Section sectionC = BitVector32.CreateSection(0xf, sectionB); BitVector32.Section sectionD = BitVector32.CreateSection(0x7, sectionC); BitVector32.Section sectionE = BitVector32.CreateSection(0x7, sectionD); BitVector32.Section sectionF = BitVector32.CreateSection(0x3, sectionE); Passing a BitVector32.Section to the indexer of the BitVector32 returns an int just mapped to the sec- tion of the bit vector. As shown next, a helper method, IntToBinaryString, retrieves a string representa- tion of the int number: Console.WriteLine("Section A: {0}", IntToBinaryString(bits2[sectionA], true)); Console.WriteLine("Section B: {0}", IntToBinaryString(bits2[sectionB], true)); c10.indd 265 30-01-2014 20:15:30 266  ❘  CHAPTER 10  Collections Console.WriteLine("Section C: {0}", IntToBinaryString(bits2[sectionC], true)); Console.WriteLine("Section D: {0}", IntToBinaryString(bits2[sectionD], true)); Console.WriteLine("Section E: {0}", IntToBinaryString(bits2[sectionE], true)); Console.WriteLine("Section F: {0}", IntToBinaryString(bits2[sectionF], true)); The method IntToBinaryString receives the bits in an integer and returns a string representation contain- ing 0 and 1. With the implementation, 32 bits of the integer are iterated through. In the iteration, if the bit is set, 1 is appended to the StringBuilder; otherwise, 0 is appended. Within the loop, a bit shift occurs to check if the next bit is set: static string IntToBinaryString(int bits, bool removeTrailingZero) { var sb = new StringBuilder(32); for (int i = 0; i < 32; i++) { if ((bits & 0x80000000) != 0) { sb.Append("1"); } else { sb.Append("0"); } bits = bits << 1; } string s = sb.ToString(); if (removeTrailingZero) { return s.TrimStart('0'); } else { return s; } } The result displays the bit representation of sections A to F, which you can now verify with the value that was passed into the bit vector: Section A: 110111101111 Section B: 10111100 Section C: 1010 Section D: 1 Section E: 111 Section F: 1 Immutable Collections If an object can change its state, it is hard to use it from multiple simultaneously running tasks. Synchronization is necessary with these collections. If an object cannot change its state, it’s a lot easier to use it from multiple threads. An object that can’t change is an immutable object. With Visual Studio 2013 Microsoft offers a new collection library: Microsoft Immutable Collections. As the name suggests, it contains immutable collection classes — collection classes that cannot be changed after they have been created. This library is available as a NuGet package and contains new collection classes in the namespace System.Collections.Immutable. You can use this library with both .NET 4.5 and .NET 4.5.1 projects. c10.indd 266 30-01-2014 20:15:31 Immutable Collections  ❘  267 Let’s start with a simple immutable string array. You can create the array with the static Create method as shown. The Create method is overloaded where other variants of this method allow passing any number of elements. In this code snippet (code file ImmutableCollectionsSample/Program.cs), an empty array is created: ImmutableArray a1 = ImmutableArray.Create(); An empty array is not very useful. The ImmutableArray type offers an Add method to add elements. However, contrary to other collection classes, the Add method does not change the immutable collection itself. Instead, a new immutable collection is returned. So after the call of the Add method, a1 is still an empty collection, and a2 is an immutable collection with one element. The Add method returns the new immutable collection: ImmutableArray a2 = a1.Add("Williams"); With this, it is possible to use this API in a fluent way and invoke one Add method after the other. The vari- able a3 now references an immutable collection containing four elements: ImmutableArray a3 = a2.Add("Ferrari").Add("Mercedes").Add("Red Bull Racing"); With each of these stages using the immutable array, the complete collections are not copied with every step. Instead, the immutable types make use of the shared state and only copy the collection when it’s necessary. However, it’s even more efficient to first fill the collection and then make it an immutable array. When some manipulation needs to take place, you can again use a mutable collection. A builder class offered by the immutable types helps with that. To see this in action, first an Account class is created that is put into the collection (code file ImmutableCollectionSample/Account.cs): public class Account { public string Name { get; set; } public decimal Amount { get; set; } } Next a List collection is created and filled with sample accounts (code file ImmutableCollectionSample/Program.cs): List accounts = new List() { new Account { Name = "Scrooge McDuck", Amount = 667377678765m }, new Account { Name = "Donald Duck", Amount = -200m }, new Account { Name = "Ludwig von Drake", Amount = 20000m }}; From the accounts collection, an immutable collection can be created with the extension method ToImmutableList. This extension method is available as soon as the namespace System.Collections .Immutable is opened: ImmutableList immutableAccounts = accounts.ToImmutableList(); The variable immutableAccounts can be enumerated like other collections. It just cannot be changed. Now, to make some changes again, you can create a builder by invoking the ToBuilder method. This method returns a collection that can be changed. In the sample code, all accounts with an amount larger c10.indd 267 30-01-2014 20:15:31 268  ❘  CHAPTER 10  Collections than 0 are removed. The original immutable collection is not changed. After the change with the builder is completed, a new immutable collection is created by invoking the ToImmutable method of the Builder. This collection is used next to output all overdrawn accounts: ImmutableList.Builder builder = immutableAccounts.ToBuilder(); for (int i = 0; i < builder.Count; i++) { Account a = builder[i]; if (a.Amount > 0) { builder.Remove(a); } } ImmutableList overdrawnAccounts = builder.ToImmutable(); foreach (var item in overdrawnAccounts) { Console.WriteLine("{0} {1}", item.Name, item.Amount); } Note  Read-only collections were previously discussed in this chapter in the section “Read-only Collections.” Read-only collections offer a read-only view on a collection. The collection can still be changed from others that access the collection without the read-only view. With immutable collections, nobody can change the collection. Note  The topics of using multiple tasks threads, and programming with asynchro- nous methods are explained in detail in Chapter 13, “Asynchronous Programming”, and Chapter 21. Concurrent Collections Since version 4 of the .NET Framework, .NET offers thread-safe collection classes within the namespace System.Collections.Concurrent. Thread-safe collections are guarded against multiple threads accessing them in conflicting ways. For thread-safe access of collections, the interface IProducerConsumerCollection is defined. The most important methods of this interface are TryAdd and TryTake. TryAdd tries to add an item to the collection, but this might fail if the collection is locked from adding items. To provide this information, the method returns a Boolean value indicating success or failure. TryTake works the same way to inform the caller about success or failure, and returns on success an item from the collection. The following list describes the collection classes from the System.Collections.Concurrent namespace and its functionality: ➤➤ ConcurrentQueue — This class is implemented with a lock-free algorithm and uses 32 item arrays that are combined in a linked list internally. Methods to access the elements of the queue are Enqueue, TryDequeue, and TryPeek. The naming of these methods is very similar to the methods of Queue that you know already, with the difference of the “Try” prefix to indicate the method call might fail. Because this class implements the interface IProducerConsumerCollection, the methods TryAdd and TryTake just invoke Enqueue and TryDequeue. c10.indd 268 30-01-2014 20:15:31 Concurrent Collections  ❘  269 ➤➤ ConcurrentStack — Very similar to ConcurrentQueue but with other item access methods, this class defines the methods Push, PushRange, TryPeek, TryPop, and TryPopRange. Internally this class uses a linked list of its items. ➤➤ ConcurrentBag — This class doesn’t define any order in which to add or take items. It uses a con- cept that maps threads to arrays used internally and thus tries to reduce locks. The methods to access elements are Add, TryPeek, and TryTake. ➤➤ ConcurrentDictionary — This is a thread-safe collection of keys and values. TryAdd, TryGetValue, TryRemove, and TryUpdate are methods to access the members in a non- blocking fashion. Because the items are based on keys and values, ConcurrentDictionary does not implement IProducerConsumerCollection. ➤➤ BlockingCollection — A collection that blocks and waits until it is possible to do the task by adding or taking the item, BlockingCollection offers an interface to add and remove items with the Add and Take methods. These methods block the thread and wait until the task becomes pos- sible. The Add method has an overload whereby you also can pass a CancellationToken. This token enables cancelling a blocking call. If you don’t want the thread to wait for an endless time, and you don’t want to cancel the call from the outside, the methods TryAdd and TryTake are offered as well, whereby you can also specify a time-out value for the maximum amount of time you would like to block the thread and wait before the call should fail. The ConcurrentXXX collection classes are thread-safe, returning false if an action is not possible with the current state of threads. You always have to check whether adding or taking the item was successful before moving on. You can’t trust the collection to always fulfill the task. BlockingCollection is a decorator to any class implementing the IProducerConsumerCollection interface and by default uses ConcurrentQueue. With the constructor you can also pass any other class that implements IProducerConsumerCollection, e.g., ConcurrentBag and ConcurrentStack. Creating Pipelines A great use for these concurrent collection classes is with pipelines. One task writes some content to a col- lection class while another task can read from the collection at the same time. The following sample application demonstrates the use of the BlockingCollection class with multiple tasks that form a pipeline. The first pipeline is shown in Figure 10-6. The task for the first stage reads file- names and adds them to a queue. While this task is running, the task for stage two can already start to read the filenames from the queue and load their content. The result is written to another queue. Stage 3 can be started at the same time to read the content from the second queue and process it. Here, the result is written to a dictionary. In this scenario, the next stage can only start when stage 3 is completed and the content is finally processed with a full result in the dictionary. The next steps are shown in Figure 10-7. Stage 4 reads from the diction- ary, converts the data, and writes it to a queue. Stage 5 adds color information to the items and puts them in another queue. The last stage displays the information. Stages 4 to 6 can run concurrently as well. Looking at the code of this sample application, the complete pipeline is managed within the method StartPipeline. Here, the collections are instantiated and passed to the various stages of the pipeline. The first stage is processed with ReadFilenamesAsync, and the second and third stages, LoadContentAsync and ProcessContentAsync, are running simultaneously. The fourth stage, however, can only start when the first three stages are completed (code file PipelineSample/Program.cs): private static async void StartPipeline() { var fileNames = new BlockingCollection(); var lines = new BlockingCollection(); var words = new ConcurrentDictionary(); var items = new BlockingCollection(); c10.indd 269 30-01-2014 20:15:31 270  ❘  CHAPTER 10  Collections FIGURE 10-6 Read Filenames Load Content Process Content Transfer Content Add Color Display Content FIGURE 10-7 var coloredItems = new BlockingCollection(); Task t1 = PipelineStages.ReadFilenamesAsync(@"../../..", fileNames); ConsoleHelper.WriteLine("started stage 1"); Task t2 = PipelineStages.LoadContentAsync(fileNames, lines); ConsoleHelper.WriteLine("started stage 2"); Task t3 = PipelineStages.ProcessContentAsync(lines, words); await Task.WhenAll(t1, t2, t3); ConsoleHelper.WriteLine("stages 1, 2, 3 completed"); c10.indd 270 30-01-2014 20:15:34 Concurrent Collections  ❘  271 Task t4 = PipelineStages.TransferContentAsync(words, items); Task t5 = PipelineStages.AddColorAsync(items, coloredItems); Task t6 = PipelineStages.ShowContentAsync(coloredItems); ConsoleHelper.WriteLine("stages 4, 5, 6 started"); await Task.WhenAll(t4, t5, t6); ConsoleHelper.WriteLine("all stages finished"); } Note  This example application makes use of tasks and the async and await key- words, which are explained in detail in Chapter 13. You can read more about threads, tasks, and synchronization in Chapter 21. File I/O is discussed in Chapter 24, “Manipulating Files and the Registry.” The example writes information to the console using the ConsoleHelper class. This class provides an easy way to change the color for console output and uses synchronization to avoid returning output with the wrong colors (code file PipelineSample/ConsoleHelper.cs): using System; namespace Wrox.ProCSharp.Collections { public class ConsoleHelper { private static object syncOutput = new object(); public static void WriteLine(string message) { lock (syncOutput) { Console.WriteLine(message); } } public static void WriteLine(string message, string color) { lock (syncOutput) { Console.ForegroundColor = (ConsoleColor)Enum.Parse( typeof(ConsoleColor), color); Console.WriteLine(message); Console.ResetColor(); } } } } Using a BlockingCollection Let’s get into the first stage of the pipeline. ReadFilenamesAsync receives a BlockingCollection where it can write its output. The implementation of this method uses an enumerator to iter- ate C# files within the specified directory and its subdirectories. The filenames are added to the BlockingCollection with the Add method. After adding filenames is completed, the CompleteAdding method is invoked to inform all readers that they should not wait for any additional items in the collection (code file PipelineSample/PipelineStages.cs): using System.Collections.Concurrent; using System.IO; using System.Linq; using System.Threading.Tasks; c10.indd 271 30-01-2014 20:15:34 272  ❘  CHAPTER 10  Collections namespace Wrox.ProCSharp.Collections { public static class PipelineStages { public static Task ReadFilenamesAsync(string path, BlockingCollection output) { return Task.Run(() => { foreach (string filename in Directory.EnumerateFiles(path, "*.cs", SearchOption.AllDirectories)) { output.Add(filename); ConsoleHelper.WriteLine(string.Format("stage 1: added {0}", filename)); } output.CompleteAdding(); }); } Note  If you have a reader that reads from a BlockingCollection at the same time a writer adds items, it is important to invoke the CompleteAdding method. Otherwise, the reader would wait for more items to arrive within the foreach loop. The next stage is to read the file and add its content to another collection , which is done from the LoadContentAsync method. This method uses the filenames passed with the input collection, opens the file, and adds all lines of the file to the output collection. With the foreach loop, the method GetConsumingEnumerable is invoked with the input blocking collection to iterate the items. It’s possible to use the input variable directly without invoking GetConsumingEnumerable, but this would only iterate the current state of the collection, and not the items that are added afterwards: public static async Task LoadContentAsync(BlockingCollection input, BlockingCollection output) { foreach (var filename in input.GetConsumingEnumerable()) { using (FileStream stream = File.OpenRead(filename)) { var reader = new StreamReader(stream); string line = null; while ((line = await reader.ReadLineAsync()) != null) { output.Add(line); ConsoleHelper.WriteLine(string.Format("stage 2: added {0}", line)); } } } output.CompleteAdding(); } Note  If a reader is reading a collection at the same time while it is filled, you need to get the enumerator of the blocking collection with the method GetConsumingEnumerable instead of iterating the collection directly. c10.indd 272 30-01-2014 20:15:35 Concurrent Collections  ❘  273 Using a ConcurrentDictionary Stage 3 is implemented in the ProcessContentAsync method. This method gets the lines from the input collection, and then splits and filters words to an output dictionary. The method AddOrIncrementValue is a helper method implemented as an extension method for dictionaries is shown next: public static Task ProcessContentAsync(BlockingCollection input, ConcurrentDictionary output) { return Task.Run(() => { foreach (var line in input.GetConsumingEnumerable()) { string[] words = line.Split(' ', ';', '\t', '{', '}', '(', ')', ':', ',', '"'); foreach (var word in words.Where(w => !string.IsNullOrEmpty(w))) { output.AddOrIncrementValue(word); ConsoleHelper.WriteLine(string.Format("stage 3: added {0}", word)); } } }); } Note  Extension methods are explained in Chapter 3, “Objects and Types.” Remember that stage 3 in the pipeline adds a word to the dictionary if it doesn’t exist yet, and increments a value in the dictionary if the word is already in there. This functionality is implemented in the extension method AddOrIncrementValue. Because the dictionary cannot be used with the BlockingCollection, there are no blocking methods that wait until adding values succeeds. Instead, TryXXX methods can be used where it’s necessary to verify if adding or updating the value succeeded. If another thread were updat- ing a value at the same time, updates can fail. The implementation makes use of TryGetValue to check if an item is already in the dictionary, TryUpdate to update a value, and TryAdd to add a value (code file PipelineSample/ConcurrentDictionaryExtensions.cs): using System.Collections.Concurrent; namespace Wrox.ProCSharp.Collections { public static class ConcurrentDictionaryExtension { public static void AddOrIncrementValue( this ConcurrentDictionary dict, string key) { bool success = false; while (!success) { int value; if (dict.TryGetValue(key, out value)) { if (dict.TryUpdate(key, value + 1, value)) { success = true; } } else { if (dict.TryAdd(key, 1)) c10.indd 273 30-01-2014 20:15:35 274  ❘  CHAPTER 10  Collections { success = true; } } } } } } Running the application with the first three stages, you’ll see output like the following, with one where the stages operating are interleaved: stage 3: added get stage 3: added set stage 3: added public stage 3: added int stage 3: added Wins stage 2: added public static class Pipeline stage 2: added { stage 2: added public static Task ReadFil stage 2: added { stage 2: added return Task.Run(() => Completing the Pipeline After the first three stages are completed, the next three stages can run in parallel again. TransferContentAsync gets the data from the dictionary, converts it to the type Info, and puts it into the output BlockingCollectiony (code file PipelineSample/PipelineStages.cs): public static Task TransferContentAsync( ConcurrentDictionary input, BlockingCollection output) { return Task.Run(() => { foreach (var word in input.Keys) { int value; if (input.TryGetValue(word, out value)) { var info = new Info { Word = word, Count = value }; output.Add(info); ConsoleHelper.WriteLine(string.Format("stage 4: added {0}", info)); } } output.CompleteAdding(); }); } The pipeline stage AddColorAsync sets the Color property of the Info type depending on the value of the Count property: public static Task AddColorAsync(BlockingCollection input, BlockingCollection output) { return Task.Run(() => { foreach (var item in input.GetConsumingEnumerable()) { if (item.Count > 40) { item.Color = "Red"; c10.indd 274 30-01-2014 20:15:35 Performance  ❘  275 } else if (item.Count > 20) { item.Color = "Yellow"; } else { item.Color = "Green"; } output.Add(item); ConsoleHelper.WriteLine(string.Format( "stage 5: added color {1} to {0}", item, item.Color)); } output.CompleteAdding(); }); } The last stage writes the resulting items to the console in the specified color: public static Task ShowContentAsync(BlockingCollection input) { return Task.Run(() => { foreach (var item in input.GetConsumingEnumerable()) { ConsoleHelper.WriteLine(string.Format("stage 6: {0}", item), item.Color); } }); } Running the application results in the output shown in Figure 10-8. FIGURE 10-8 Performance Many collection classes offer the same functionality as others; for example, SortedList offers nearly the same features as SortedDictionary. However, often there’s a big difference in performance. Whereas one collection consumes less memory, the other collection class is faster with retrieval of elements. The MSDN documentation often provides performance hints about methods of the collection, giving you information about the time the operation requires in big-O notation: O(1) O(log n) O(n) c10.indd 275 30-01-2014 20:15:35 276  ❘  CHAPTER 10  Collections O(1) means that the time this operation needs is constant no matter how many items are in the collection. For example, the ArrayList has an Add method with O(1) behavior. No matter how many elements are in the list, it always takes the same amount of time when adding a new element to the end of the list. The Count property provides the number of items, so it is easy to find the end of the list. O(n) means it takes the worst case time of N to perform an operation on the collection. The Add method of ArrayList can be an O(n) operation if a reallocation of the collection is required. Changing the capacity causes the list to be copied, and the time for the copy increases linearly with every element. O(log n) means that the time needed for the operation increases with every element in the collection, but the increase of time for each element is not linear but logarithmic. SortedDictionary has O(log n) behavior for inserting operations inside the collection; SortedList has O(n) behavior for the same functionality. Here, SortedDictionary is a lot faster because it is more efficient to insert elements into a tree structure than into a list. The following table lists collection classes and their performance for different actions such as adding, insert- ing, and removing items. Using this table you can select the best collection class for the purpose of your use. The left column lists the collection class. The Add column gives timing information about adding items to the collection. The List and the HashSet classes define Add methods to add items to the collec- tion. With other collection classes use a different method to add elements to the collection; for example, the Stack class defines a Push method, and the Queue class defines an Enqueue method. You can find this information in the table as well. If there are multiple big-O values in a cell, the reason is because if a collection needs to be resized, resizing takes a while. For example, with the List class, adding items needs O(1). If the capacity of the collection is not large enough and the collection needs to be resized, the resize requires O(n) time. The larger the col- lection, the longer the resize operation takes. It’s best to avoid resizes by setting the capacity of the collection to a value that can hold all the elements. If the table cell contents is n/a, the operation is not applicable with this collection type. Collection Add Insert Remove Item Sort Find List O(1) or O(n) if the collec- tion must be resized O(n) O(n) O(1) O (n log n), worst case O(n ^ 2) O(n) Stack Push, O(1) or O(n) if the stack must be resized n/a Pop, O(1) n/a n/a n/a Queue Enqueue, O(1) or O(n) if the queue must be resized n/a Dequeue, O(1) n/a n/a n/a HashSet O(1) or O(n) if the set must be resized Add O(1) or O(n) O(1) n/a n/a n/a SortedSet O(1) or O(n) if the set must be resized Add O(1) or O(n) O(1) n/a n/a n/a LinkedList AddLast O(1) Add After O(1) O(1) n/a n/a O(n) c10.indd 276 30-01-2014 20:15:35 Summary  ❘  277 Dictionary O(1) or O(n) n/a O(1) O(1) n/a n/a SortedDictionary O(log n) n/a O(log n) O(log n) n/a n/a SortedList O(n) for unsorted data, O(log n) for end of list, O(n) if resize is needed n/a O(n) O(log n) to read/write, O(log n) if the key is in the list, O(n) if the key is not in the list n/a n/a Summary This chapter took a look at working with different kinds of collections. Arrays are fixed in size, but you can use lists for dynamically growing collections. For accessing elements on a first-in, first-out basis, there’s a queue; and you can use a stack for last-in, first-out operations. Linked lists allow for fast inser- tion and removal of elements but are slow for searching. With keys and values, you can use dictionaries, which are fast for searching and inserting elements. Sets are useful for unique items and can be ordered (SortedSet) or not ordered (HashSet). ObservableCollection raises events when items change in the list. You’ve also looked at several interfaces and classes in this chapter, including how to use them for access- ing and sorting collections. Finally, you looked at some specialized collections, such as BitArray and BitVector32, which are optimized for working with a collection of bits. Chapter 11 gives you details about Language Integrated Query (LINQ). c10.indd 277 30-01-2014 20:15:35 c10.indd 278 30-01-2014 20:15:35 11 Language Integrated Query WHAT’S IN THIS CHAPTER? ➤ Traditional queries across objects using List ➤ Extension methods ➤ LINQ query operators ➤ Parallel LINQ ➤ Expression trees WRox.CoM CoDE DoWNLoADS FoR THIS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤ LINQ Intro ➤ Enumerable Sample ➤ Parallel LINQ ➤ Expression Trees LINQ ovERvIEW LINQ (Language Integrated Query) integrates query syntax inside the C# programming language, making it possible to access different data sources with the same syntax. LINQ accomplishes this by offering an abstraction layer. This chapter describes the core principles of LINQ and the language extensions for C# that make the C# LINQ Query possible. NoTE For details about using LINQ across the database, you should read Chapter 33, “ADO.NET Entity Framework.” For information about querying XML data, read Chapter 34, “Manipulating XML,” after reading this chapter. c11.indd 279 30-01-2014 20:16:11 280  ❘  CHAPTER 11  Language Integrated Query This chapter starts with a simple LINQ query before diving into the full potential of LINQ. The C# language offers integrated query language that is converted to method calls. This section shows you what the conversion looks like so you can use all the possibilities of LINQ. Lists and Entities The LINQ queries in this chapter are performed on a collection containing Formula-1 champions from 1950 to 2011. This data needs to be prepared with entity classes and lists. For the entities, the type Racer is defined. Racer defines several properties and an overloaded ToString method to display a racer in a string format. This class implements the interface IFormattable to support different variants of format strings, and the interface IComparable, which can be used to sort a list of racers based on the LastName. For more advanced queries, the class Racer contains not only single-value properties such as FirstName, LastName, Wins, Country, and Starts, but also multivalue properties such as Cars and Years. The Years property lists all the years of the championship title. Some racers have won more than one title. The Cars property is used to list all the cars used by the driver during the title years (code file DataLib/Racer.cs): using System; using System.Collections.Generic; namespace Wrox.ProCSharp.LINQ { [Serializable] public class Racer: IComparable, IFormattable { public Racer(string firstName, string lastName, string country, int starts, int wins) : this(firstName, lastName, country, starts, wins, null, null) { } public Racer(string firstName, string lastName, string country, int starts, int wins, IEnumerable years, IEnumerable cars) { this.FirstName = firstName; this.LastName = lastName; this.Country = country; this.Starts = starts; this.Wins = wins; this.Years = new List(years); this.Cars = new List(cars); } public string FirstName {get; set;} public string LastName {get; set;} public int Wins {get; set;} public string Country {get; set;} public int Starts {get; set;} public IEnumerable Cars { get; private set; } public IEnumerable Years { get; private set; } public override string ToString() { return String.Format("{0} {1}", FirstName, LastName); } public int CompareTo(Racer other) { if (other == null) return -1; c11.indd 280 30-01-2014 20:16:11 LINQ Overview  ❘  281 return string.Compare(this.LastName, other.LastName); } public string ToString(string format) { return ToString(format, null); } public string ToString(string format, IFormatProvider formatProvider) { switch (format) { case null: case "N": return ToString(); case "F": return FirstName; case "L": return LastName; case "C": return Country; case "S": return Starts.ToString(); case "W": return Wins.ToString(); case "A": return String.Format("{0} {1}, {2}; starts: {3}, wins: {4}", FirstName, LastName, Country, Starts, Wins); default: throw new FormatException(String.Format( "Format {0} not supported", format)); } } } } A second entity class is Team. This class just contains the name and an array of years for constructor championships. Similar to a driver championship, there’s a constructor championship for the best team of a year (code file DataLib/Team.cs): [Serializable] public class Team { public Team(string name, params int[] years) { this.Name = name; this.Years = new List(years); } public string Name { get; private set; } public IEnumerable Years { get; private set; } } The class Formula1 returns a list of racers in the method GetChampions. The list is filled with all Formula-1 champions from the years 1950 to 2011 (code file DataLib/Formula1.cs): using System.Collections.Generic; namespace Wrox.ProCSharp.LINQ { public static class Formula1 c11.indd 281 30-01-2014 20:16:11 282  ❘  CHAPTER 11  Language Integrated Query { private static List racers; public static IList GetChampions() { if (racers == null) { racers = new List(40); racers.Add(new Racer("Nino", "Farina", "Italy", 33, 5, new int[] { 1950 }, new string[] { "Alfa Romeo" })); racers.Add(new Racer("Alberto", "Ascari", "Italy", 32, 10, new int[] { 1952, 1953 }, new string[] { "Ferrari" })); racers.Add(new Racer("Juan Manuel", "Fangio", "Argentina", 51, 24, new int[] { 1951, 1954, 1955, 1956, 1957 }, new string[] { "Alfa Romeo", "Maserati", "Mercedes", "Ferrari" })); racers.Add(new Racer("Mike", "Hawthorn", "UK", 45, 3, new int[] { 1958 }, new string[] { "Ferrari" })); racers.Add(new Racer("Phil", "Hill", "USA", 48, 3, new int[] { 1961 }, new string[] { "Ferrari" })); racers.Add(new Racer("John", "Surtees", "UK", 111, 6, new int[] { 1964 }, new string[] { "Ferrari" })); racers.Add(new Racer("Jim", "Clark", "UK", 72, 25, new int[] { 1963, 1965 }, new string[] { "Lotus" })); racers.Add(new Racer("Jack", "Brabham", "Australia", 125, 14, new int[] { 1959, 1960, 1966 }, new string[] { "Cooper", "Brabham" })); racers.Add(new Racer("Denny", "Hulme", "New Zealand", 112, 8, new int[] { 1967 }, new string[] { "Brabham" })); racers.Add(new Racer("Graham", "Hill", "UK", 176, 14, new int[] { 1962, 1968 }, new string[] { "BRM", "Lotus" })); racers.Add(new Racer("Jochen", "Rindt", "Austria", 60, 6, new int[] { 1970 }, new string[] { "Lotus" })); racers.Add(new Racer("Jackie", "Stewart", "UK", 99, 27, new int[] { 1969, 1971, 1973 }, new string[] { "Matra", "Tyrrell" })); //... return racers; } } } } Where queries are done across multiple lists, the GetConstructorChampions method that follows returns the list of all constructor championships (these championships have been around since 1958): private static List teams; public static IList GetContructorChampions() { if (teams == null) { teams = new List() { new Team("Vanwall", 1958), new Team("Cooper", 1959, 1960), new Team("Ferrari", 1961, 1964, 1975, 1976, 1977, 1979, 1982, 1983, 1999, 2000, 2001, 2002, 2003, 2004, 2007, 2008), new Team("BRM", 1962), new Team("Lotus", 1963, 1965, 1968, 1970, 1972, 1973, 1978), new Team("Brabham", 1966, 1967), new Team("Matra", 1969), new Team("Tyrrell", 1971), c11.indd 282 30-01-2014 20:16:11 LINQ Overview  ❘  283 new Team("McLaren", 1974, 1984, 1985, 1988, 1989, 1990, 1991, 1998), new Team("Williams", 1980, 1981, 1986, 1987, 1992, 1993, 1994, 1996, 1997), new Team("Benetton", 1995), new Team("Renault", 2005, 2006), new Team("Brawn GP", 2009), new Team("Red Bull Racing", 2010, 2011) }; } return teams; } LINQ Query Using these prepared lists and entities, you can do a LINQ query — for example, a query to get all world champions from Brazil sorted by the highest number of wins. To accomplish this you could use methods of the List class; e.g., the FindAll and Sort methods. However, using LINQ there’s a simpler syntax as soon as you get used to it (code file LINQIntro/Program.cs): private static void LinqQuery() { var query = from r in Formula1.GetChampions() where r.Country == "Brazil" orderby r.Wins descending select r; foreach (Racer r in query) { Console.WriteLine("{0:A}", r); } } The result of this query shows world champions from Brazil ordered by number of wins: Ayrton Senna, Brazil; starts: 161, wins: 41 Nelson Piquet, Brazil; starts: 204, wins: 23 Emerson Fittipaldi, Brazil; starts: 143, wins: 14 The statement from r in Formula1.GetChampions() where r.Country == "Brazil" orderby r.Wins descending select r; is a LINQ query. The clauses from, where, orderby, descending, and select are predefined keywords in this query. The query expression must begin with a from clause and end with a select or group clause. In between you can optionally use where, orderby, join, let, and additional from clauses. Note  The variable query just has the LINQ query assigned to it. The query is not performed by this assignment, but rather as soon as the query is accessed using the foreach loop. This is discussed in more detail later in the section “Deferred Query Execution.” c11.indd 283 30-01-2014 20:16:12 284  ❘  CHAPTER 11  Language Integrated Query Extension Methods The compiler converts the LINQ query to invoke method calls instead of the LINQ query. LINQ offers various extension methods for the IEnumerable interface, so you can use the LINQ query across any collection that implements this interface. An extension method is defined as a static method whose first parameter defines the type it extends, and it is declared in a static class. Extension methods make it possible to write a method to a class that doesn’t already offer the method at first. You can also add a method to any class that implements a specific interface, so multiple classes can make use of the same implementation. For example, wouldn’t you like to have a Foo method with the String class? The String class is sealed, so it is not possible to inherit from this class; but you can create an extension method, as shown in the following code: public static class StringExtension { public static void Foo(this string s) { Console.WriteLine("Foo invoked for {0}", s); } } An extension method is defined as a static method where the first parameter defines the type it extends and it is declared in a static class. The Foo method extends the string class, as is defined with the first parameter. For differentiating extension methods from normal static methods, the extension method also requires the this keyword with the first parameter. Indeed, it is now possible to use the Foo method with the string type: string s = "Hello"; s.Foo(); The result shows Foo invoked for Hello in the console, because Hello is the string passed to the Foo method. This might appear to be breaking object-oriented rules because a new method is defined for a type without changing the type or deriving from it. However, this is not the case. The extension method cannot access private members of the type it extends. Calling an extension method is just a new syntax for invoking a static method. With the string you can get the same result by calling the method Foo this way: string s = "Hello"; StringExtension.Foo(s); To invoke the static method, write the class name followed by the method name. Extension methods are a different way to invoke static methods. You don’t have to supply the name of the class where the static method is defined. Instead, because of the parameter type the static method is selected by the compiler. You just have to import the namespace that contains the class to get the Foo extension method in the scope of the String class. One of the classes that define LINQ extension methods is Enumerable in the namespace System.Linq. You just have to import the namespace to open the scope of the extension methods of this class. A sample implementation of the Where extension method is shown in the following code. The first parameter of the Where method that includes the this keyword is of type IEnumerable. This enables the Where method to be used with every type that implements IEnumerable. A few examples of types that implement this interface are arrays and List. The second parameter is a Func delegate that references a method that returns a Boolean value and requires a parameter of type T. This predicate is invoked within the implementation to examine whether the item from the IEnumerable source should be added into the c11.indd 284 30-01-2014 20:16:12 LINQ Overview  ❘  285 destination collection. If the method is referenced by the delegate, the yield return statement returns the item from the source to the destination: public static IEnumerable Where( this IEnumerable source, Func predicate) { foreach (TSource item in source) if (predicate(item)) yield return item; } Because Where is implemented as a generic method, it works with any type that is contained in a collection. Any collection implementing IEnumerable is supported. Note  The extension methods here are defined in the namespace System.Linq in the assembly System.Core. Now it’s possible to use the extension methods Where, OrderByDescending, and Select from the class Enumerable. Because each of these methods returns IEnumerable, it is possible to invoke one method after the other by using the previous result. With the arguments of the extension methods, anonymous methods that define the implementation for the delegate parameters are used (code file LINQIntro/Program.cs): static void ExtensionMethods() { var champions = new List(Formula1.GetChampions()); IEnumerable brazilChampions = champions.Where(r => r.Country == "Brazil"). OrderByDescending(r => r.Wins). Select(r => r); foreach (Racer r in brazilChampions) { Console.WriteLine("{0:A}", r); } } Deferred Query Execution When the query expression is defined during runtime, the query does not run. The query runs when the items are iterated. Let’s have a look once more at the extension method Where. This extension method makes use of the yield return statement to return the elements where the predicate is true. Because the yield return statement is used, the compiler creates an enumerator and returns the items as soon as they are accessed from the enumeration: public static IEnumerable Where(this IEnumerable source, Func predicate) { foreach (T item in source) { if (predicate(item)) { yield return item; } } } c11.indd 285 30-01-2014 20:16:12 286  ❘  CHAPTER 11  Language Integrated Query This has a very interesting and important effect. In the following example a collection of string elements is created and filled with first names. Next, a query is defined to get all names from the collection whose first letter is J. The collection should also be sorted. The iteration does not happen when the query is defined. Instead, the iteration happens with the foreach statement, where all items are iterated. Only one element of the collection fulfills the requirements of the where expression by starting with the letter J: Juan. After the iteration is done and Juan is written to the console, four new names are added to the collection. Then the iteration is done again: var names = new List { "Nino", "Alberto", "Juan", "Mike", "Phil" }; var namesWithJ = from n in names where n.StartsWith("J") orderby n select n; Console.WriteLine("First iteration"); foreach (string name in namesWithJ) { Console.WriteLine(name); } Console.WriteLine(); names.Add("John"); names.Add("Jim"); names.Add("Jack"); names.Add("Denny"); Console.WriteLine("Second iteration"); foreach (string name in namesWithJ) { Console.WriteLine(name); } Because the iteration does not happen when the query is defined, but does happen with every foreach, changes can be seen, as the output from the application demonstrates: First iteration Juan Second iteration Jack Jim John Juan Of course, you also must be aware that the extension methods are invoked every time the query is used within an iteration. Most of the time this is very practical, because you can detect changes in the source data. However, sometimes this is impractical. You can change this behavior by invoking the extension methods ToArray, ToList, and the like. In the following example, you can see that ToList iterates through the collection immediately and returns a collection implementing IList. The returned list is then iterated through twice; in between iterations, the data source gets new names: var names = new List { "Nino", "Alberto", "Juan", "Mike", "Phil" }; var namesWithJ = (from n in names where n.StartsWith("J") orderby n select n).ToList(); Console.WriteLine("First iteration"); c11.indd 286 30-01-2014 20:16:12 Standard Query Operators  ❘  287 foreach (string name in namesWithJ) { Console.WriteLine(name); } Console.WriteLine(); names.Add("John"); names.Add("Jim"); names.Add("Jack"); names.Add("Denny"); Console.WriteLine("Second iteration"); foreach (string name in namesWithJ) { Console.WriteLine(name); } The result indicates that in between the iterations the output stays the same although the collection values have changed: First iteration Juan Second iteration Juan Standard Query Operators Where, OrderByDescending, and Select are only a few of the query operators defined by LINQ. The LINQ query defines a declarative syntax for the most common operators. There are many more query operators available with the Enumerable class. The following table lists the standard query operators defined by the Enumerable class. Standard Query Operators Description Where OfType Filtering operators define a restriction to the elements returned. With the Where query operator you can use a predicate; for example, a Lambda expression that returns a bool. OfType filters the elements based on the type and returns only the elements of the type TResult. Select SelectMany Projection operators are used to transform an object into a new object of a different type. Select and SelectMany define a projection to select values of the result based on a selector function. OrderBy ThenBy OrderByDescending ThenByDescending Reverse Sorting operators change the order of elements returned. OrderBy sorts values in ascending order; OrderByDescending sorts values in descending order. ThenBy and ThenByDescending operators are used for a secondary sort if the first sort gives similar results. Reverse reverses the elements in the collection. Join GroupJoin Join operators are used to combine collections that might not be directly related to each other. With the Join operator a join of two collections based on key selector functions can be done. This is similar to the JOIN you know from SQL. The GroupJoin operator joins two collections and groups the results. GroupBy ToLookup Grouping operators put the data into groups. The GroupBy operator groups elements with a common key. ToLookup groups the elements by creating a one-to-many dictionary. (continues) c11.indd 287 30-01-2014 20:16:12 288  ❘  CHAPTER 11  Language Integrated Query Standard Query Operators Description Any All Contains Quantifier operators return a Boolean value if elements of the sequence satisfy a specific condition. Any, All, and Contains are quantifier operators. Any determines if any element in the collection satisfies a predicate function; All determines if all elements in the collection satisfy a predicate. Contains checks whether a specific element is in the collection. Take Skip TakeWhile SkipWhile Partitioning operators return a subset of the collection. Take, Skip, TakeWhile, and SkipWhile are partitioning operators. With these, you get a partial result. With Take, you have to specify the number of elements to take from the collection; Skip ignores the specified number of elements and takes the rest. TakeWhile takes the elements as long as a condition is true. Distinct Union Intersect Except Zip Set operators return a collection set. Distinct removes duplicates from a collection. With the exception of Distinct, the other set operators require two collections. Union returns unique elements that appear in either of the two collections. Intersect returns elements that appear in both collections. Except returns elements that appear in just one collection. Zip combines two collections into one. First FirstOrDefault Last LastOrDefault ElementAt ElementAtOrDefault Single SingleOrDefault Element operators return just one element. First returns the first element that satisfies a condition. FirstOrDefault is similar to First, but it returns a default value of the type if the element is not found. Last returns the last element that satisfies a condition. With ElementAt, you specify the position of the element to return. Single returns only the one element that satisfies a condition. If more than one element satisfies the condition, an exception is thrown. Count Sum Min Max Average Aggregate Aggregate operators compute a single value from a collection. With aggregate operators, you can get the sum of all values, the number of all elements, the element with the lowest or highest value, an average number, and so on. ToArray AsEnumerable ToList ToDictionary Cast Conversion operators convert the collection to an array: IEnumerable, IList, IDictionary, and so on. Empty Range Repeat Generation operators return a new sequence. The collection is empty using the Empty operator; Range returns a sequence of numbers, and Repeat returns a collection with one repeated value. The following sections provide examples demonstrating how to use these operators. (continued) c11.indd 288 30-01-2014 20:16:12 Standard Query Operators  ❘  289 Filtering This section looks at some examples for a query. With the where clause, you can combine multiple expressions — for example, get only the racers from Brazil and Austria who won more than 15 races. The result type of the expression passed to the where clause just needs to be of type bool: var racers = from r in Formula1.GetChampions() where r.Wins > 15 && (r.Country == "Brazil" || r.Country == "Austria") select r; foreach (var r in racers) { Console.WriteLine("{0:A}", r); } Starting the program with this LINQ query returns Niki Lauda, Nelson Piquet, and Ayrton Senna, as shown here: Niki Lauda, Austria, Starts: 173, Wins: 25 Nelson Piquet, Brazil, Starts: 204, Wins: 23 Ayrton Senna, Brazil, Starts: 161, Wins: 41 Not all queries can be done with the LINQ query syntax, and not all extension methods are mapped to LINQ query clauses. Advanced queries require using extension methods. To better understand complex queries with extension methods, it’s good to see how simple queries are mapped. Using the extension methods Where and Select produces a query very similar to the LINQ query done before: var racers = Formula1.GetChampions(). Where(r => r.Wins > 15 && (r.Country == "Brazil" || r.Country == "Austria")). Select(r => r); Filtering with Index One scenario in which you can’t use the LINQ query is an overload of the Where method. With an overload of the Where method, you can pass a second parameter that is the index. The index is a counter for every result returned from the filter. You can use the index within the expression to do some calculation based on the index. In the following example, the index is used within the code that is called by the Where extension method to return only racers whose last name starts with A if the index is even (code file EnumerableSample/Program.cs): var racers = Formula1.GetChampions(). Where((r, index) => r.LastName.StartsWith("A") && index % 2 != 0); foreach (var r in racers) { Console.WriteLine("{0:A}", r); } The racers with last names beginning with the letter A are Alberto Ascari, Mario Andretti, and Fernando Alonso. Because Mario Andretti is positioned within an index that is odd, he is not in the result: Alberto Ascari, Italy; starts: 32, wins: 10 Fernando Alonso, Spain; starts: 177, wins: 27 c11.indd 289 30-01-2014 20:16:13 290  ❘  CHAPTER 11  Language Integrated Query Type Filtering For filtering based on a type you can use the OfType extension method. Here the array data contains both string and int objects. Using the extension method OfType, passing the string class to the generic parameter returns only the strings from the collection (code file EnumerableSample/Program.cs): object[] data = { "one", 2, 3, "four", "five", 6 }; var query = data.OfType(); foreach (var s in query) { Console.WriteLine(s); } Running this code, the strings one, four, and five are displayed: one four five Compound from If you need to do a filter based on a member of the object that itself is a sequence, you can use a compound from. The Racer class defines a property Cars, where Cars is a string array. For a filter of all racers who were champions with a Ferrari, you can use the LINQ query shown next. The first from clause accesses the Racer objects returned from Formula1.GetChampions. The second from clause accesses the Cars property of the Racer class to return all cars of type string. Next the cars are used with the where clause to filter only the racers who were champions with a Ferrari (code file EnumerableSample/Program.cs): var ferrariDrivers = from r in Formula1.GetChampions() from c in r.Cars where c == "Ferrari" orderby r.LastName select r.FirstName + " " + r.LastName; If you are curious about the result of this query, following are all Formula-1 champions driving a Ferrari: Alberto Ascari Juan Manuel Fangio Mike Hawthorn Phil Hill Niki Lauda Kimi Räikkönen Jody Scheckter Michael Schumacher John Surtees The C# compiler converts a compound from clause with a LINQ query to the SelectMany extension method. SelectMany can be used to iterate a sequence of a sequence. The overload of the SelectMany method that is used with the example is shown here: public static IEnumerable SelectMany ( this IEnumerable source, Func> collectionSelector, Func resultSelector); The first parameter is the implicit parameter that receives the sequence of Racer objects from the GetChampions method. The second parameter is the collectionSelector delegate where the inner c11.indd 290 30-01-2014 20:16:13 Standard Query Operators  ❘  291 sequence is defined. With the lambda expression r => r.Cars, the collection of cars should be returned. The third parameter is a delegate that is now invoked for every car and receives the Racer and Car objects. The lambda expression creates an anonymous type with a Racer and a Car property. As a result of this SelectMany method, the hierarchy of racers and cars is flattened and a collection of new objects of an anonymous type for every car is returned. This new collection is passed to the Where method so that only the racers driving a Ferrari are filtered. Finally, the OrderBy and Select methods are invoked: var ferrariDrivers = Formula1.GetChampions(). SelectMany(r => r.Cars, (r, c) => new { Racer = r, Car = c }). Where(r => r.Car == "Ferrari"). OrderBy(r => r.Racer.LastName). Select(r => r.Racer.FirstName + " " + r.Racer.LastName); Resolving the generic SelectMany method to the types that are used here, the types are resolved as follows. In this case the source is of type Racer, the filtered collection is a string array, and of course the name of the anonymous type that is returned is not known and is shown here as TResult: public static IEnumerable SelectMany ( this IEnumerable source, Func> collectionSelector, Func resultSelector); Because the query was just converted from a LINQ query to extension methods, the result is the same as before. Sorting To sort a sequence, the orderby clause was used already. This section reviews the earlier example, now with the orderby descending clause. Here the racers are sorted based on the number of wins as specified by the key selector in descending order (code file EnumerableSample/Program.cs): var racers = from r in Formula1.GetChampions() where r.Country == "Brazil" orderby r.Wins descending select r; The orderby clause is resolved to the OrderBy method, and the orderby descending clause is resolved to the OrderByDescending method: var racers = Formula1.GetChampions(). Where(r => r.Country == "Brazil"). OrderByDescending(r => r.Wins). Select(r => r); The OrderBy and OrderByDescending methods return IOrderedEnumerable. This interface derives from the interface IEnumerable but contains an additional method, CreateOrdered Enumerable. This method is used for further ordering of the sequence. If two items are the same based on the key selector, ordering can continue with the ThenBy and ThenByDescending methods. These methods require an IOrderedEnumerable to work on but return this interface as well. Therefore, you can add any number of ThenBy and ThenByDescending methods to sort the collection. Using the LINQ query, you just add all the different keys (with commas) for sorting to the orderby clause. In the next example, the sort of all racers is done first based on country, next on last name, and finally c11.indd 291 30-01-2014 20:16:13 292  ❘  CHAPTER 11  Language Integrated Query on first name. The Take extension method that is added to the result of the LINQ query is used to return the first 10 results: var racers = (from r in Formula1.GetChampions() orderby r.Country, r.LastName, r.FirstName select r).Take(10); The sorted result is shown here: Argentina: Fangio, Juan Manuel Australia: Brabham, Jack Australia: Jones, Alan Austria: Lauda, Niki Austria: Rindt, Jochen Brazil: Fittipaldi, Emerson Brazil: Piquet, Nelson Brazil: Senna, Ayrton Canada: Villeneuve, Jacques Finland: Hakkinen, Mika Doing the same with extension methods makes use of the OrderBy and ThenBy methods: var racers = Formula1.GetChampions(). OrderBy(r => r.Country). ThenBy(r => r.LastName). ThenBy(r => r.FirstName). Take(10); Grouping To group query results based on a key value, the group clause can be used. Now the Formula-1 champions should be grouped by country, and the number of champions within a country should be listed. The clause group r by r.Country into g groups all the racers based on the Country property and defines a new identifier g that can be used later to access the group result information. The result from the group clause is ordered based on the extension method Count that is applied on the group result; and if the count is the same, the ordering is done based on the key. This is the country because this was the key used for grouping. The where clause filters the results based on groups that have at least two items, and the select clause creates an anonymous type with the Country and Count properties (code file EnumerableSample/Program.cs): var countries = from r in Formula1.GetChampions() group r by r.Country into g orderby g.Count() descending, g.Key where g.Count() >= 2 select new { Country = g.Key, Count = g.Count() }; foreach (var item in countries) { Console.WriteLine("{0, -10} {1}", item.Country, item.Count); } The result displays the collection of objects with the Country and Count properties: UK 10 Brazil 3 c11.indd 292 30-01-2014 20:16:13 Standard Query Operators  ❘  293 Finland 3 Australia 2 Austria 2 Germany 2 Italy 2 USA 2 Doing the same with extension methods, the groupby clause is resolved to the GroupBy method. What’s interesting with the declaration of the GroupBy method is that it returns an enumeration of objects implementing the IGrouping interface. The IGrouping interface defines the Key property, so you can access the key of the group after defining the call to this method: public static IEnumerable> GroupBy( this IEnumerable source, Func keySelector); The group r by r.Country into g clause is resolved to GroupBy(r => r.Country) and returns the group sequence. The group sequence is first ordered by the OrderByDecending method, then by the ThenBy method. Next, the Where and Select methods that you already know are invoked: var countries = Formula1.GetChampions(). GroupBy(r => r.Country). OrderByDescending(g => g.Count()). ThenBy(g => g.Key). Where(g => g.Count() >= 2). Select(g => new { Country = g.Key, Count = g.Count() }); Grouping with Nested Objects If the grouped objects should contain nested sequences, you can do that by changing the anonymous type created by the select clause. With this example, the returned countries should contain not only the properties for the name of the country and the number of racers, but also a sequence of the names of the racers. This sequence is assigned by using an inner from/in clause assigned to the Racers property. The inner from clause is using the g group to get all racers from the group, order them by last name, and create a new string based on the first and last name (code file EnumerableSample/ Program.cs): var countries = from r in Formula1.GetChampions() group r by r.Country into g orderby g.Count() descending, g.Key where g.Count() >= 2 select new { Country = g.Key, Count = g.Count(), Racers = from r1 in g orderby r1.LastName select r1.FirstName + " " + r1.LastName }; foreach (var item in countries) { Console.WriteLine("{0, -10} {1}", item.Country, item.Count); foreach (var name in item.Racers) { Console.Write("{0}; ", name); } Console.WriteLine(); } c11.indd 293 30-01-2014 20:16:13 294  ❘  CHAPTER 11  Language Integrated Query The output now lists all champions from the specified countries: UK 10 Jenson Button; Jim Clark; Lewis Hamilton; Mike Hawthorn; Graham Hill; Damon Hill; James Hunt; Nigel Mansell; Jackie Stewart; John Surtees; Brazil 3 Emerson Fittipaldi; Nelson Piquet; Ayrton Senna; Finland 3 Mika Hakkinen; Kimi Raikkonen; Keke Rosberg; Australia 2 Jack Brabham; Alan Jones; Austria 2 Niki Lauda; Jochen Rindt; Germany 2 Michael Schumacher; Sebastian Vettel; Italy 2 Alberto Ascari; Nino Farina; USA 2 Mario Andretti; Phil Hill; Inner Join You can use the join clause to combine two sources based on specific criteria. First, however, let’s get two lists that should be joined. With Formula-1, there are drivers and a constructors championships. The drivers are returned from the method GetChampions, and the constructors are returned from the method GetConstructorChampions. It would be interesting to get a list by year in which every year lists the driver and the constructor champions. To do this, the first two queries for the racers and the teams are defined (code file EnumerableSample/ Program.cs): var racers = from r in Formula1.GetChampions() from y in r.Years select new { Year = y, Name = r.FirstName + " " + r.LastName }; var teams = from t in Formula1.GetContructorChampions() from y in t.Years select new { Year = y, Name = t.Name }; Using these two queries, a join is done based on the year of the driver champion and the year of the team champion with the join clause. The select clause defines a new anonymous type containing Year, Racer, and Team properties: var racersAndTeams = (from r in racers join t in teams on r.Year equals t.Year select new { r.Year, Champion = r.Name, Constructor = t.Name }).Take(10); c11.indd 294 30-01-2014 20:16:13 Standard Query Operators  ❘  295 Console.WriteLine("Year World Champion\t Constructor Title"); foreach (var item in racersAndTeams) { Console.WriteLine("{0}: {1,-20} {2}", item.Year, item.Champion, item.Constructor); } Of course you can also combine this to just one LINQ query, but that’s a matter of taste: var racersAndTeams = (from r in from r1 in Formula1.GetChampions() from yr in r1.Years select new { Year = yr, Name = r1.FirstName + " " + r1.LastName } join t in from t1 in Formula1.GetContructorChampions() from yt in t1.Years select new { Year = yt, Name = t1.Name } on r.Year equals t.Year orderby t.Year select new { Year = r.Year, Racer = r.Name, Team = t.Name }).Take(10); The output displays data from the anonymous type for the first 10 years in which both a drivers and constructor championship took place: Year World Champion Constructor Title 1958: Mike Hawthorn Vanwall 1959: Jack Brabham Cooper 1960: Jack Brabham Cooper 1961: Phil Hill Ferrari 1962: Graham Hill BRM 1963: Jim Clark Lotus 1964: John Surtees Ferrari 1965: Jim Clark Lotus 1966: Jack Brabham Brabham 1967: Denny Hulme Brabham Left Outer Join The output from the previous join sample started with the year 1958 — the first year when both the drivers’ and constructor championship started. The drivers’ championship started earlier, in the year 1950. With an inner join, results are returned only when matching records are found. To get a result with all the years included, a left outer join can be used. A left outer join returns all the elements in the left sequence even when no match is found in the right sequence. The earlier LINQ query is changed to a left outer join. A left outer join is defined with the join clause together with the DefaultIfEmpty method. If the left side of the query (the racers) does not have a c11.indd 295 30-01-2014 20:16:13 296  ❘  CHAPTER 11  Language Integrated Query matching constructor champion, the default value for the right side is defined by the DefaultIfEmpty method (code file EnumerableSample/Program.cs): var racersAndTeams = (from r in racers join t in teams on r.Year equals t.Year into rt from t in rt.DefaultIfEmpty() orderby r.Year select new { Year = r.Year, Champion = r.Name, Constructor = t == null ? “no constructor championship” : t.Name }).Take(10); Running the application with this query, the output starts with the year 1950 as shown here: Year Champion Constructor Title 1950: Nino Farina no constructor championship 1951: Juan Manuel Fangio no constructor championship 1952: Alberto Ascari no constructor championship 1953: Alberto Ascari no constructor championship 1954: Juan Manuel Fangio no constructor championship 1955: Juan Manuel Fangio no constructor championship 1956: Juan Manuel Fangio no constructor championship 1957: Juan Manuel Fangio no constructor championship 1958: Mike Hawthorn Vanwall 1959: Jack Brabham Cooper Group Join A left outer join makes use of a group join together with the into clause. It uses partly the same syntax as the group join. The group join just doesn’t need the DefaultIfEmpty method. With a group join, two independent sequences can be joined, whereby one sequence contains a list of items for one element of the other sequence. The following example uses two independent sequences. One is the list of champions that you already know from previous examples. The second sequence is a collection of Championship types. The Championship type is shown in the next code snippet. This class contains the year of the championship and the racers with the first, second, and third position of the year with the properties Year, First, Second, and Third (code file DataLib/Championship.cs): public class Championship { public int Year { get; set; } public string First { get; set; } public string Second { get; set; } public string Third { get; set; } } The collection of championships is returned from the method GetChampionships as shown in the following code snippet (code file DataLib/Formula1.cs): private static List championships; public static IEnumerable GetChampionships() { if (championships == null) { championships = new List(); championships.Add(new Championship c11.indd 296 30-01-2014 20:16:14 Standard Query Operators  ❘  297 { Year = 1950, First = "Nino Farina", Second = "Juan Manuel Fangio", Third = "Luigi Fagioli" }); championships.Add(new Championship { Year = 1951, First = "Juan Manuel Fangio", Second = "Alberto Ascari", Third = "Froilan Gonzalez" }); //... The list of champions should be combined with the list of racers that are found within the first three positions in every year of championships, and the results for every year should be displayed. The information that should be shown is defined with the RacerInfo class, as shown here (code file EnumerableSample/RacerInfo.cs): public class RacerInfo { public int Year { get; set; } public int Position { get; set; } public string FirstName { get; set; } public string LastName { get; set; } } With a join statement the racers from both lists can be combined. Because in the list of championships every item contains three racers, this list needs to be flattened first. One way to do this is by using the SelectMany method. SelectMany makes use of a lambda expression that returns a list of three items for every item in the list. Within the implementation of the lambda expression, because the RacerInfo contains the FirstName and the LastName properties, and the collection received just contains only a name with First, Second, and Third properties, the string needs to be divided. This is done with the help of the extension methods FirstName and SecondName (code file EnumerableSample/ Program.cs): var racers = Formula1.GetChampionships() .SelectMany(cs => new List() { new RacerInfo { Year = cs.Year, Position = 1, FirstName = cs.First.FirstName(), LastName = cs.First.LastName() }, new RacerInfo { Year = cs.Year, Position = 2, FirstName = cs.Second.FirstName(), LastName = cs.Second.LastName() }, new RacerInfo { Year = cs.Year, Position = 3, FirstName = cs.Third.FirstName(), LastName = cs.Third.LastName() } }); c11.indd 297 30-01-2014 20:16:14 298  ❘  CHAPTER 11  Language Integrated Query The extension methods FirstName and SecondName just use the last blank character to split up the string: public static class StringExtension { public static string FirstName(this string name) { int ix = name.LastIndexOf(' '); return name.Substring(0, ix); } public static string LastName(this string name) { int ix = name.LastIndexOf(' '); return name.Substring(ix + 1); } } Now the two sequences can be joined. Formula1.GetChampions returns a list of Racers, and the racers variable returns the list of RacerInfo that contains the year, the result, and the names of racers. It’s not enough to compare the items from these two collections by using the last name. Sometimes a racer and his father can be found in the list (e.g., Damon Hill and Graham Hill), so it’s necessary to compare the items by both FirstName and LastName. This is done by creating a new anonymous type for both lists. Using the into clause, the result from the second collection is put into the variable yearResults. yearResults is created for every racer in the first collection and contains the results of the matching first name and last name from the second collection. Finally, with the LINQ query a new anonymous type is created that contains the needed information: var q = (from r in Formula1.GetChampions() join r2 in racers on new { FirstName = r.FirstName, LastName = r.LastName } equals new { FirstName = r2.FirstName, LastName = r2.LastName } into yearResults select new { FirstName = r.FirstName, LastName = r.LastName, Wins = r.Wins, Starts = r.Starts, Results = yearResults }); foreach (var r in q) { Console.WriteLine("{0} {1}", r.FirstName, r.LastName); foreach (var results in r.Results) { Console.WriteLine("{0} {1}.", results.Year, results.Position); } } The last results from the foreach loop are shown next. Lewis Hamilton has been twice among the top three, 2007 as second and 2008 as first. Jenson Button is found three times, 2004, 2009, and 2011; and Sebastian Vettel was world champion two times and had the second position in 2009: c11.indd 298 30-01-2014 20:16:14 Standard Query Operators  ❘  299 Lewis Hamilton 2007 2. 2008 1. Jenson Button 2004 3. 2009 1. 2011 2. Sebastian Vettel 2009 2. 2010 1. 2011 1. Set Operations The extension methods Distinct, Union, Intersect, and Except are set operations. The following example creates a sequence of Formula-1 champions driving a Ferrari and another sequence of Formula-1 champions driving a McLaren, and then determines whether any driver has been a champion driving both of these cars. Of course, that’s where the Intersect extension method can help. First, you need to get all champions driving a Ferrari. This uses a simple LINQ query with a compound from to access the property Cars that’s returning a sequence of string objects (code file EnumerableSample/ Program.cs): var ferrariDrivers = from r in Formula1.GetChampions() from c in r.Cars where c == "Ferrari" orderby r.LastName select r; Now the same query with a different parameter of the where clause is needed to get all McLaren racers. It’s not a good idea to write the same query again. One option is to create a method in which you can pass the parameter car: private static IEnumerable GetRacersByCar(string car) { return from r in Formula1.GetChampions() from c in r.Cars where c == car orderby r.LastName select r; } However, because the method wouldn’t be needed in other places, defining a variable of a delegate type to hold the LINQ query is a good approach. The variable racersByCar needs to be of a delegate type that requires a string parameter and returns IEnumerable, similar to the method implemented earlier. To do this, several generic Func<> delegates are defined, so you do not need to declare your own delegate. A lambda expression is assigned to the variable racersByCar. The left side of the lambda expression defines a car variable of the type that is the first generic parameter of the Func delegate (a string). The right side defines the LINQ query that uses the parameter with the where clause: Func> racersByCar = car => from r in Formula1.GetChampions() from c in r.Cars where c == car orderby r.LastName select r; c11.indd 299 30-01-2014 20:16:14 300  ❘  CHAPTER 11  Language Integrated Query Now you can use the Intersect extension method to get all racers who won the championship with a Ferrari and a McLaren: Console.WriteLine("World champion with Ferrari and McLaren"); foreach (var racer in racersByCar("Ferrari").Intersect( racersByCar("McLaren"))) { Console.WriteLine(racer); } The result is just one racer, Niki Lauda: World champion with Ferrari and McLaren Niki Lauda Note  The set operations compares the objects by invoking the GetHashCode and Equals methods of the entity class. For custom comparisons, you can also pass an object that implements the interface IEqualityComparer. In the preceding example here, the GetChampions method always returns the same objects, so the default comparison works. If that’s not the case, the set methods offer overloads in which a comparison can be defined. Zip The Zip method is new since .NET 4 and enables you to merge two related sequences into one with a predicate function. First, two related sequences are created, both with the same filtering (country Italy) and ordering. For merging this is important, as item 1 from the first collection is merged with item 1 from the second collection, item 2 with item 2, and so on. In case the count of the two sequences is different, Zip stops when the end of the smaller collection is reached. The items in the first collection have a Name property and the items in the second collection have LastName and Starts properties. Using the Zip method on the collection racerNames requires the second collection racerNamesAndStarts as the first parameter. The second parameter is of type Func. This parameter is implemented as a lambda expression and receives the elements of the first collection with the parameter first, and the elements of the second collection with the parameter second. The implementation creates and returns a string containing the Name property of the first element and the Starts property of the second element (code file EnumerableSample/Program.cs): var racerNames = from r in Formula1.GetChampions() where r.Country == "Italy" orderby r.Wins descending select new { Name = r.FirstName + " " + r.LastName }; var racerNamesAndStarts = from r in Formula1.GetChampions() where r.Country == "Italy" orderby r.Wins descending select new { LastName = r.LastName, c11.indd 300 30-01-2014 20:16:14 Standard Query Operators  ❘  301 Starts = r.Starts }; var racers = racerNames.Zip(racerNamesAndStarts, (first, second) => first.Name + ", starts: " + second.Starts); foreach (var r in racers) { Console.WriteLine(r); } The result of this merge is shown here: Alberto Ascari, starts: 32 Nino Farina, starts: 33 Partitioning Partitioning operations such as the extension methods Take and Skip can be used for easy paging — for example, to display just 5 racers on the first page, and continue with the next 5 on the following pages. With the LINQ query shown here, the extension methods Skip and Take are added to the end of the query. The Skip method first ignores a number of items calculated based on the page size and the actual page number; the Take method then takes a number of items based on the page size (code file EnumerableSample/Program.cs): int pageSize = 5; int numberPages = (int)Math.Ceiling(Formula1.GetChampions().Count() / (double)pageSize); for (int page = 0; page < numberPages; page++) { Console.WriteLine("Page {0}", page); var racers = (from r in Formula1.GetChampions() orderby r.LastName, r.FirstName select r.FirstName + " " + r.LastName). Skip(page * pageSize).Take(pageSize); foreach (var name in racers) { Console.WriteLine(name); } Console.WriteLine(); } Here is the output of the first three pages: Page 0 Fernando Alonso Mario Andretti Alberto Ascari Jack Brabham Jenson Button Page 1 Jim Clark Juan Manuel Fangio Nino Farina c11.indd 301 30-01-2014 20:16:14 302  ❘  CHAPTER 11  Language Integrated Query Emerson Fittipaldi Mika Hakkinen Page 2 Lewis Hamilton Mike Hawthorn Damon Hill Graham Hill Phil Hill Paging can be extremely useful with Windows or web applications, showing the user only a part of the data. Note  Note an important behavior of this paging mechanism: because the query is done with every page, changing the underlying data affects the results. New objects are shown as paging continues. Depending on your scenario, this can be advantageous to your application. If this behavior is not what you need, you can do the paging not over the original data source but by using a cache that maps to the original data. With the TakeWhile and SkipWhile extension methods you can also pass a predicate to retrieve or skip items based on the result of the predicate. Aggregate Operators The aggregate operators such as Count, Sum, Min, Max, Average, and Aggregate do not return a sequence but a single value instead. The Count extension method returns the number of items in the collection. In the following example, the Count method is applied to the Years property of a Racer to filter the racers and return only those who won more than three championships. Because the same count is needed more than once in the same query, a variable numberYears is defined by using the let clause (code file EnumerableSample/Program.cs): var query = from r in Formula1.GetChampions() let numberYears = r.Years.Count() where numberYears >= 3 orderby numberYears descending, r.LastName select new { Name = r.FirstName + " " + r.LastName, TimesChampion = numberYears }; foreach (var r in query) { Console.WriteLine("{0} {1}", r.Name, r.TimesChampion); } The result is shown here: Michael Schumacher 7 Juan Manuel Fangio 5 Alain Prost 4 Jack Brabham 3 Niki Lauda 3 Nelson Piquet 3 Ayrton Senna 3 Jackie Stewart 3 c11.indd 302 30-01-2014 20:16:14 Standard Query Operators  ❘  303 The Sum method summarizes all numbers of a sequence and returns the result. In the next example, Sum is used to calculate the sum of all race wins for a country. First the racers are grouped based on country; then, with the new anonymous type created, the Wins property is assigned to the sum of all wins from a single country: var countries = (from c in from r in Formula1.GetChampions() group r by r.Country into c select new { Country = c.Key, Wins = (from r1 in c select r1.Wins).Sum() } orderby c.Wins descending, c.Country select c).Take(5); foreach (var country in countries) { Console.WriteLine("{0} {1}", country.Country, country.Wins); } The most successful countries based on the Formula-1 race champions are as follows: UK 167 Germany 112 Brazil 78 France 51 Finland 42 The methods Min, Max, Average, and Aggregate are used in the same way as Count and Sum. Min returns the minimum number of the values in the collection, and Max returns the maximum number. Average calculates the average number. With the Aggregate method you can pass a lambda expression that performs an aggregation of all the values. Conversion Operators In this chapter you’ve already seen that query execution is deferred until the items are accessed. Using the query within an iteration, the query is executed. With a conversion operator, the query is executed immediately and the result is returned in an array, a list, or a dictionary. In the next example, the ToList extension method is invoked to immediately execute the query and put the result into a List (code file EnumerableSample/Program.cs): List racers = (from r in Formula1.GetChampions() where r.Starts > 150 orderby r.Starts descending select r).ToList(); foreach (var racer in racers) { Console.WriteLine("{0} {0:S}", racer); } It’s not that simple to get the returned objects into the list. For example, for fast access from a car to a racer within a collection class, you can use the new class Lookup. c11.indd 303 30-01-2014 20:16:15 304  ❘  CHAPTER 11  Language Integrated Query Note  The Dictionary class supports only a single value for a key. With the class Lookup from the namespace System.Linq, you can have multiple values for a single key. These classes are covered in detail in Chapter 10, “Collections.” Using the compound from query, the sequence of racers and cars is flattened, and an anonymous type with the properties Car and Racer is created. With the lookup that is returned, the key should be of type string referencing the car, and the value should be of type Racer. To make this selection, you can pass a key and an element selector to one overload of the ToLookup method. The key selector references the Car property, and the element selector references the Racer property: var racers = (from r in Formula1.GetChampions() from c in r.Cars select new { Car = c, Racer = r }).ToLookup(cr => cr.Car, cr => cr.Racer); if (racers.Contains("Williams")) { foreach (var williamsRacer in racers["Williams"]) { Console.WriteLine(williamsRacer); } } The result of all “Williams” champions accessed using the indexer of the Lookup class is shown here: Alan Jones Keke Rosberg Nigel Mansell Alain Prost Damon Hill Jacques Villeneuve In case you need to use a LINQ query over an untyped collection, such as the ArrayList, you can use the Cast method. In the following example, an ArrayList collection that is based on the Object type is filled with Racer objects. To make it possible to define a strongly typed query, you can use the Cast method: var list = new System.Collections.ArrayList(Formula1.GetChampions() as System.Collections.ICollection); var query = from r in list.Cast() where r.Country == "USA" orderby r.Wins descending select r; foreach (var racer in query) { Console.WriteLine("{0:A}", racer); } Generation Operators The generation operators Range, Empty, and Repeat are not extension methods, but normal static methods that return sequences. With LINQ to Objects, these methods are available with the Enumerable class. c11.indd 304 30-01-2014 20:16:15 Parallel LINQ  ❘  305 Have you ever needed a range of numbers filled? Nothing is easier than using the Range method. This method receives the start value with the first parameter and the number of items with the second parameter: var values = Enumerable.Range(1, 20); foreach (var item in values) { Console.Write("{0} ", item); } Console.WriteLine(); Note  The Range method does not return a collection filled with the values as defined. This method does a deferred query execution similar to the other methods. It returns a RangeEnumerator that simply does a yield return with the values incremented. Of course, the result now looks like this: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 You can combine the result with other extension methods to get a different result — for example, using the Select extension method: var values = Enumerable.Range(1, 20).Select(n => n * 3); The Empty method returns an iterator that does not return values. This can be used for parameters that require a collection for which you can pass an empty collection. The Repeat method returns an iterator that returns the same value a specific number of times. Parallel LINQ The class ParallelEnumerable in the System.Linq namespace to splits the work of queries across multiple threads. Although the Enumerable class defines extension methods to the IEnumerable interface, most extension methods of the ParallelEnumerable class are extensions for the class ParallelQuery. One important exception is the AsParallel method, which extends IEnumerable and returns ParallelQuery, so a normal collection class can be queried in a parallel manner. Parallel Queries To demonstrate Parallel LINQ (PLINQ), a large collection is needed. With small collections you won’t see any effect when the collection fits inside the CPU’s cache. In the following code, a large int collection is filled with random values (code file ParallelLinqSample/Program.cs): static IEnumerable SampleData() { const int arraySize = 100000000; var r = new Random(); return Enumerable.Range(0, arraySize).Select(x => r.Next(140)).ToList(); } Now you can use a LINQ query to filter the data, do some calculations, and get an average of the filtered data. The query defines a filter with the where clause to summarize only the items with values < 20, and c11.indd 305 30-01-2014 20:16:15 306  ❘  CHAPTER 11  Language Integrated Query then the aggregation function sum is invoked. The only difference to the LINQ queries you’ve seen so far is the call to the AsParallel method: var res = (from x in data.AsParallel() where Math.Log(x) < 4 select x).Average(); Like the LINQ queries shown already, the compiler changes the syntax to invoke the methods AsParallel, Where, Select, and Average. AsParallel is defined with the ParallelEnumerable class to extend the IEnumerable interface, so it can be called with a simple array. AsParallel returns ParallelQuery. Because of the returned type, the Where method chosen by the compiler is ParallelEnumerable.Where instead of Enumerable.Where. In the following code, the Select and Average methods are from ParallelEnumerable as well. In contrast to the implementation of the Enumerable class, with the ParallelEnumerable class the query is partitioned so that multiple threads can work on the query. The collection can be split into multiple parts whereby different threads work on each part to filter the remaining items. After the partitioned work is completed, merging must occur to get the summary result of all parts: var res = data.AsParallel().Where(x => Math.Log(x) < 4). Select(x => x).Average(); Running this code starts the task manager so you can confirm that all CPUs of your system are busy. If you remove the AsParallel method, multiple CPUs might not be used. Of course, if you don’t have multiple CPUs on your system, then don’t expect to see an improvement with the parallel version. Partitioners The AsParallel method is an extension not only to the IEnumerable interface, but also to the Partitioner class. With this you can influence the partitions to be created. The Partitioner class is defined within the namespace System.Collections.Concurrent and has different variants. The Create method accepts arrays or objects implementing IList. Depending on that, as well as on the parameter loadBalance , which is of type Boolean and available with some overloads of the method, a different partitioner type is returned. For arrays, .NET 4 includes DynamicPartitionerFor Array and StaticPartitionerForArray, both of which derive from the abstract base class OrderablePartitioner. In the following example, the code from the “Parallel Queries” section is changed to manually create a partitioner instead of relying on the default one: var result = (from x in Partitioner.Create(data, true).AsParallel() where Math.Log(x) < 4 select x).Average(); You can also influence the parallelism by invoking the methods WithExecutionMode and WithDegreeOfParallelism. With WithExecutionMode you can pass a value of ParallelExecutionMode, which can be Default or ForceParallelism. By default, Parallel LINQ avoids parallelism with high overhead. With the method WithDegreeOfParallelism you can pass an integer value to specify the maximum number of tasks that should run in parallel. This is useful if not all CPU cores should be used by the query. Cancellation .NET offers a standard way to cancel long-running tasks, and this is also true for Parallel LINQ. To cancel a long-running query, you can add the method WithCancellation to the query and pass a CancellationToken to the parameter. The CancellationToken is created from the c11.indd 306 30-01-2014 20:16:15 Uploaded by [StormRG] Expression Trees  ❘  307 CancellationTokenSource. The query is run in a separate thread where the exception of type OperationCanceledException is caught. This exception is fired if the query is cancelled. From the main thread the task can be cancelled by invoking the Cancel method of the CancellationTokenSource: var cts = new CancellationTokenSource(); Task.Factory.StartNew(() => { try { var res = (from x in data.AsParallel().WithCancellation(cts.Token) where Math.Log(x) < 4 select x).Average(); Console.WriteLine("query finished, sum: {0}", res); } catch (OperationCanceledException ex) { Console.WriteLine(ex.Message); } }); Console.WriteLine("query started"); Console.Write("cancel? "); string input = Console.ReadLine(); if (input.ToLower().Equals("y")) { // cancel! cts.Cancel(); } Note  You can read more about cancellation and the CancellationToken in Chapter 21, “Tasks, Threads, and Synchronization.” Expression Trees With LINQ to Objects, the extension methods require a delegate type as parameter; this way, a lambda expression can be assigned to the parameter. Lambda expressions can also be assigned to parameters of type Expression. The C# compiler defines different behavior for lambda expressions depending on the type. If the type is Expression, the compiler creates an expression tree from the lambda expression and stores it in the assembly. The expression tree can be analyzed during runtime and optimized for querying against the data source. Let’s turn to a query expression that was used previously (code file ExpressionTreeSample/Program.cs): var brazilRacers = from r in racers where r.Country == "Brazil" orderby r.Wins select r; The preceding query expression uses the extension methods Where, OrderBy, and Select. The Enumerable class defines the Where extension method with the delegate type Func as parameter predicate: public static IEnumerable Where( this IEnumerable source, Func predicate); c11.indd 307 30-01-2014 20:16:15 308  ❘  CHAPTER 11  Language Integrated Query This way, the lambda expression is assigned to the predicate. Here, the lambda expression is similar to an anonymous method, as explained earlier: Func predicate = r => r.Country == "Brazil"; The Enumerable class is not the only class for defining the Where extension method. The Where extension method is also defined by the class Queryable. This class has a different definition of the Where extension method: public static IQueryable Where( this IQueryable source, Expression> predicate); Here, the lambda expression is assigned to the type Expression, which behaves differently: Expression> predicate = r => r.Country == "Brazil"; Instead of using delegates, the compiler emits an expression tree to the assembly. The expression tree can be read during runtime. Expression trees are built from classes derived from the abstract base class Expression. The Expression class is not the same as Expression. Some of the expression classes that inherit from Expression include BinaryExpression, ConstantExpression, InvocationExpression, LambdaExpression, NewExpression, NewArrayExpression, TernaryExpression, UnaryExpression, and more. The compiler creates an expression tree resulting from the lambda expression. For example, the lambda expression r.Country == "Brazil" makes use of ParameterExpression, MemberExpression, ConstantExpression, and MethodCallExpression to create a tree and store the tree in the assembly. This tree is then used during runtime to create an optimized query to the underlying data source. The method DisplayTree is implemented to display an expression tree graphically on the console. In the following example, an Expression object can be passed, and depending on the expression type some information about the expression is written to the console. Depending on the type of the expression, DisplayTree is called recursively: Note  This method does not deal with all expression types, only the types that are used with the following example expression. private static void DisplayTree(int indent, string message, Expression expression) { string output = String.Format("{0} {1} ! NodeType: {2}; Expr: {3} ", "".PadLeft(indent, '>'), message, expression.NodeType, expression); indent++; switch (expression.NodeType) { case ExpressionType.Lambda: Console.WriteLine(output); LambdaExpression lambdaExpr = (LambdaExpression)expression; foreach (var parameter in lambdaExpr.Parameters) { DisplayTree(indent, "Parameter", parameter); } DisplayTree(indent, "Body", lambdaExpr.Body); break; c11.indd 308 30-01-2014 20:16:16 Expression Trees  ❘  309 case ExpressionType.Constant: ConstantExpression constExpr = (ConstantExpression)expression; Console.WriteLine("{0} Const Value: {1}", output, constExpr.Value); break; case ExpressionType.Parameter: ParameterExpression paramExpr = (ParameterExpression)expression; Console.WriteLine("{0} Param Type: {1}", output, paramExpr.Type.Name); break; case ExpressionType.Equal: case ExpressionType.AndAlso: case ExpressionType.GreaterThan: BinaryExpression binExpr = (BinaryExpression)expression; if (binExpr.Method != null) { Console.WriteLine("{0} Method: {1}", output, binExpr.Method.Name); } else { Console.WriteLine(output); } DisplayTree(indent, "Left", binExpr.Left); DisplayTree(indent, "Right", binExpr.Right); break; case ExpressionType.MemberAccess: MemberExpression memberExpr = (MemberExpression)expression; Console.WriteLine("{0} Member Name: {1}, Type: {2}", output, memberExpr.Member.Name, memberExpr.Type.Name); DisplayTree(indent, "Member Expr", memberExpr.Expression); break; default: Console.WriteLine(); Console.WriteLine("{0} {1}", expression.NodeType, expression.Type.Name); break; } } The expression that is used for showing the tree is already well known. It’s a lambda expression with a Racer parameter, and the body of the expression takes racers from Brazil only if they have won more than six races: Expression> expression = r => r.Country == "Brazil" && r.Wins > 6; DisplayTree(0, "Lambda", expression); Looking at the tree result, you can see from the output that the lambda expression consists of a Parameter and an AndAlso node type. The AndAlso node type has an Equal node type to the left and a GreaterThan node type to the right. The Equal node type to the left of the AndAlso node type has a MemberAccess node type to the left and a Constant node type to the right, and so on: Lambda! NodeType: Lambda; Expr: r => ((r.Country == "Brazil") AndAlso (r.Wins > 6)) > Parameter! NodeType: Parameter; Expr: r Param Type: Racer > Body! NodeType: AndAlso; Expr: ((r.Country == "Brazil") AndAlso (r.Wins > 6)) >> Left! NodeType: Equal; Expr: (r.Country == "Brazil") Method: op_Equality >>> Left! NodeType: MemberAccess; Expr: r.Country c11.indd 309 30-01-2014 20:16:16 310  ❘  CHAPTER 11  Language Integrated Query Member Name: Country, Type: String >>>> Member Expr! NodeType: Parameter; Expr: r Param Type: Racer >>> Right! NodeType: Constant; Expr: "Brazil" Const Value: Brazil >> Right! NodeType: GreaterThan; Expr: (r.Wins > 6) >>> Left! NodeType: MemberAccess; Expr: r.Wins Member Name: Wins, Type: Int32 >>>> Member Expr! NodeType: Parameter; Expr: r Param Type: Racer >>> Right! NodeType: Constant; Expr: 6 Const Value: 6 Examples where the Expression type is used are with the ADO.NET Entity Framework and the client provider for WCF Data Services. These technologies define methods with Expression parameters. This way the LINQ provider accessing the database can create a runtime-optimized query by reading the expressions to get the data from the database. LINQ Providers .NET includes several LINQ providers. A LINQ provider implements the standard query operators for a specific data source. LINQ providers might implement more extension methods than are defined by LINQ, but the standard operators must at least be implemented. LINQ to XML implements additional methods that are particularly useful with XML, such as the methods Elements, Descendants, and Ancestors defined by the class Extensions in the System.Xml.Linq namespace. Implementation of the LINQ provider is selected based on the namespace and the type of the first parameter. The namespace of the class that implements the extension methods must be opened; otherwise, the extension class is not in scope. The parameter of the Where method defined by LINQ to Objects and the Where method defined by LINQ to Entities is different. The Where method of LINQ to Objects is defined with the Enumerable class: public static IEnumerable Where( this IEnumerable source, Func predicate); Inside the System.Linq namespace is another class that implements the operator Where. This implementation is used by LINQ to Entities. You can find the implementation in the class Queryable: public static IQueryable Where( this IQueryable source, Expression> predicate); Both of these classes are implemented in the System.Core assembly in the System.Linq namespace. How does the compiler select what method to use, and what’s the magic with the Expression type? The lambda expression is the same regardless of whether it is passed with a Func parameter or an Expression> parameter—only the compiler behaves differently. The selection is done based on the source parameter. The method that matches best based on its parameters is chosen by the compiler. The CreateQuery method of the ObjectContext class that is defined by ADO.NET Entity Framework returns an ObjectQuery object that implements IQueryable, and thus the Entity Framework uses the Where method of the Queryable class. Summary This chapter described and demonstrated the LINQ query and the language constructs on which the query is based, such as extension methods and lambda expressions. You’ve looked at the various LINQ query operators — not only for filtering and ordering of data sources, but also for partitioning, grouping, doing conversions, joins, and so on. With Parallel LINQ, you’ve seen how longer queries can easily be parallelized. c11.indd 310 30-01-2014 20:16:16 Summary  ❘  311 Another important concept of this chapter is the expression tree. Expression trees enable building the query to the data source at runtime because the tree is stored in the assembly. You can read about its great advantages in Chapter 33, “ADO.NET Entity Framework.” LINQ is a very in-depth topic, and you can see Chapters 33 and 34, “Manipulating XML,” for more information. Other third-party providers are also available for download, such as LINQ to MySQL, LINQ to Amazon, LINQ to Flickr, LINQ to LDAP, and LINQ to SharePoint. No matter what data source you have, with LINQ you can use the same query syntax. c11.indd 311 30-01-2014 20:16:16 c11.indd 312 30-01-2014 20:16:16 Dynamic Language Extensions WHAT’S in THiS CHAPTER? ➤ Understanding the Dynamic Language Runtime ➤ The dynamic type ➤ The DLR ScriptRuntime ➤ Creating dynamic objects with DynamicObject and ExpandoObject WRox.Com CoDE DoWnLoADS FoR THiS CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤ DLRHost ➤ Dynamic ➤ DynamicFileReader ➤ ErrorExample The growth of languages such as Ruby and Python, and the increased use of JavaScript, have intensifi ed interest in dynamic programming. In previous versions of the .NET Framework, the var keyword and anonymous methods started C# down the “dynamic” road. In version 4, the dynamic type was added. Although C# is still a statically typed language, these additions give it the dynamic capabilities that some developers are looking for. In this chapter, you’ll look at the dynamic type and the rules for using it. You’ll also see what an implementation of DynamicObject looks like and how it can be used. ExpandoObject, which is the frameworks implementation of DynamicObject, will also be covered. DynAmiC LAnguAgE RunTimE The dynamic capabilities of C# 4 are part of the dynamic language runtime (DLR). The DLR is a set of services that is added to the common language runtime (CLR) to enable the addition of dynamic languages such as Ruby and Python. It also enables C# to take on some of the same dynamic capabilities that these dynamic languages have. 12 c12.indd 313 30-01-2014 20:17:01 314  ❘  CHAPTER 12  Dynamic Language Extensions There is a version of the DLR that is open source and resides on the CodePlex website. This same version is included with the .NET 4.5 Framework, with some additional support for language implementers. In the .NET Framework, the DLR is found in the System.Dynamic namespace as well as a few additional classes in the System.Runtime.CompilerServices namespace. IronRuby and IronPython, which are open-source versions of the Ruby and Python languages, use the DLR. Silverlight also uses the DLR. It’s possible to add scripting capabilities to your applications by hosting the DLR. The scripting runtime enables you to pass variables to and from the script. The Dynamic Type The dynamic type enables you to write code that bypasses compile-time type checking. The compiler will assume that whatever operation is defined for an object of type dynamic is valid. If that operation isn’t valid, the error won’t be detected until runtime. This is shown in the following example: class Program { static void Main(string[] args) { var staticPerson = new Person(); dynamic dynamicPerson = new Person(); staticPerson.GetFullName("John", "Smith"); dynamicPerson.GetFullName("John", "Smith"); } } class Person { public string FirstName { get; set; } public string LastName { get; set; } public string GetFullName() { return string.Concat(FirstName, " ", LastName); } } This example will not compile because of the call to staticPerson.GetFullName. There isn’t a method on the Person object that takes two parameters, so the compiler raises the error. If that line of code were to be commented out, the example would compile. If executed, a runtime error would occur. The exception that is raised is RuntimeBinderException. The RuntimeBinder is the object in the runtime that evaluates the call to determine whether Person really does support the method that was called. Binding is discussed later in the chapter. Unlike the var keyword, an object that is defined as dynamic can change type during runtime. Remember that when the var keyword is used, the determination of the object’s type is delayed. Once the type is defined, it can’t be changed. Not only can you change the type of a dynamic object, you can change it many times. This differs from casting an object from one type to another. When you cast an object, you are creating a new object with a different but compatible type. For example, you cannot cast an int to a Person object. In the following example, you can see that if the object is a dynamic object, you can change it from int to Person: dynamic dyn; dyn = 100; Console.WriteLine(dyn.GetType()); Console.WriteLine(dyn); c12.indd 314 30-01-2014 20:17:01 The Dynamic Type  ❘  315 dyn = "This is a string"; Console.WriteLine(dyn.GetType()); Console.WriteLine(dyn); dyn = new Person() { FirstName = "Bugs", LastName = "Bunny" }; Console.WriteLine(dyn.GetType()); Console.WriteLine("{0} {1}", dyn.FirstName, dyn.LastName); Executing this code would show that the dyn object actually changes type from System.Int32 to System .String to Person. If dyn had been declared as an int or string, the code would not have compiled. Note a couple of limitations to the dynamic type. A dynamic object does not support extension methods. Nor can anonymous functions (lambda expressions) be used as parameters to a dynamic method call, so LINQ does not work well with dynamic objects. Most LINQ calls are extension methods, and lambda expressions are used as arguments to those extension methods. Dynamic Behind the Scenes So what’s going on behind the scenes to make this happen? C# is still a statically typed language. That hasn’t changed. Take a look at the IL (Intermediate Language) that’s generated when the dynamic type is used. First, this is the example C# code that you’re looking at: using System; namespace DeCompile { class Program { static void Main(string[] args) { StaticClass staticObject = new StaticClass(); DynamicClass dynamicObject = new DynamicClass(); Console.WriteLine(staticObject.IntValue); Console.WriteLine(dynamicObject.DynValue); Console.ReadLine(); } } class StaticClass { public int IntValue = 100; } class DynamicClass { public dynamic DynValue = 100; } } You have two classes, StaticClass and DynamicClass. StaticClass has a single field that returns an int. DynamicClass has a single field that returns a dynamic object. The Main method just creates these objects and prints out the value that the methods return. Simple enough. Now comment out the references to the DynamicClass in Main like this: static void Main(string[] args) { StaticClass staticObject = new StaticClass(); //DynamicClass dynamicObject = new DynamicClass(); c12.indd 315 30-01-2014 20:17:01 316  ❘  CHAPTER 12  Dynamic Language Extensions Console.WriteLine(staticObject.IntValue); //Console.WriteLine(dynamicObject.DynValue); Console.ReadLine(); } Using the ildasm tool (discussed in Chapter 19, “Assemblies”), you can look at the IL that is generated for the Main method: .method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 26 (0x1a) .maxstack 1 .locals init ([0] class DeCompile.StaticClass staticObject) IL_0000: nop IL_0001: newobj instance void DeCompile.StaticClass::.ctor() IL_0006: stloc.0 IL_0007: ldloc.0 IL_0008: ldfld int32 DeCompile.StaticClass::IntValue IL_000d: call void [mscorlib]System.Console::WriteLine(int32) IL_0012: nop IL_0013: call string [mscorlib]System.Console::ReadLine() IL_0018: pop IL_0019: ret } // end of method Program::Main Without going into the details of IL but just looking at this section of code, you can still pretty much tell what’s going on. Line 0001, the StaticClass constructor, is called. Line 0008 calls the IntValue field of StaticClass. The next line writes out the value. Now comment out the StaticClass references and uncomment the DynamicClass references: static void Main(string[] args) { //StaticClass staticObject = new StaticClass(); DynamicClass dynamicObject = new DynamicClass(); Console.WriteLine(staticObject.IntValue); //Console.WriteLine(dynamicObject.DynValue); Console.ReadLine(); } Compile the application again and this is what is generated: .method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 121 (0x79) .maxstack 9 .locals init ([0] class DeCompile.DynamicClass dynamicObject, [1] class [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo[] CS$0$0000) IL_0000: nop IL_0001: newobj instance void DeCompile.DynamicClass::.ctor() IL_0006: stloc.0 IL_0007: ldsfld class [System.Core]System.Runtime.CompilerServices.CallSite'1 > DeCompile.Program/'
o__SiteContainer0'::'<>p__Site1' c12.indd 316 30-01-2014 20:17:01 The Dynamic Type  ❘  317 IL_000c: brtrue.s IL_004d IL_000e: ldc.i4.0 IL_000f: ldstr "WriteLine" IL_0014: ldtoken DeCompile.Program IL_0019: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle (valuetype [mscorlib]System.RuntimeTypeHandle) IL_001e: ldnull IL_001f: ldc.i4.2 IL_0020: newarr [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder.CSharpArgumentInfo IL_0025: stloc.1 IL_0026: ldloc.1 IL_0027: ldc.i4.0 IL_0028: ldc.i4.s 33 IL_002a: ldnull IL_002b: newobj instance void [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder .CSharpArgumentInfo::.ctor(valuetype [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder .CSharpArgumentInfoFlags, string) IL_0030: stelem.ref IL_0031: ldloc.1 IL_0032: ldc.i4.1 IL_0033: ldc.i4.0 IL_0034: ldnull IL_0035: newobj instance void [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder .CSharpArgumentInfo::.ctor(valuetype [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder .CSharpArgumentInfoFlags, string) IL_003a: stelem.ref IL_003b: ldloc.1 IL_003c: newobj instance void [Microsoft.CSharp]Microsoft.CSharp.RuntimeBinder .CSharpInvokeMemberBinder::.ctor(valuetype Microsoft.CSharp]Microsoft.CSharp .RuntimeBinder.CSharpCallFlags, string) class [mscorlib]System.Type, class [mscorlib]System.Collections.Generic.IEnumerable'1 , class [mscorlib]System.Collections.Generic.IEnumerable'1 ) IL_0041: call class [System.Core]System.Runtime.CompilerServices.CallSite'1 class [System.Core]System.Runtime.CompilerServices.CallSite'1 >::Create(class [System.Core]System.Runtime.CompilerServices.CallSiteBinder) IL_0046: stsfld class [System.Core]System.Runtime.CompilerServices.CallSite'1 > DeCompile.Program/'
o__SiteContainer0'::'<>p__Site1' IL_004b: br.s IL_004d IL_004d: ldsfld class [System.Core]System.Runtime.CompilerServices.CallSite'1 > DeCompile.Program/'
o__SiteContainer0'::'<>p__Site1' IL_0052: ldfld !0 class [System.Core]System.Runtime.CompilerServices.CallSite'1 >::Target IL_0057: ldsfld class [System.Core]System.Runtime.CompilerServices.CallSite'1 > c12.indd 317 30-01-2014 20:17:01 318  ❘  CHAPTER 12  Dynamic Language Extensions DeCompile.Program/'
o__SiteContainer0'::'<>p__Site1' IL_005c: ldtoken [mscorlib]System.Console IL_0061: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle (valuetype [mscorlib]System.RuntimeTypeHandle) IL_0066: ldloc.0 IL_0067: ldfld object DeCompile.DynamicClass::DynValue IL_006c: callvirt instance void class [mscorlib]System.Action'3 ::Invoke(!0,!1,!2) IL_0071: nop IL_0072: call string [mscorlib]System.Console::ReadLine() IL_0077: pop IL_0078: ret } // end of method Program::Main It’s safe to say that the C# compiler is doing a little extra work to support the dynamic type. Looking at the generated code, you can see references to System.Runtime.CompilerServices.CallSite and System .Runtime.CompilerServices.CallSiteBinder. The CallSite is a type that handles the lookup at runtime. When a call is made on a dynamic object at runtime, something has to check that object to determine whether the member really exists. The call site caches this information so the lookup doesn’t have to be performed repeatedly. Without this process, performance in looping structures would be questionable. After the CallSite does the member lookup, the CallSiteBinder is invoked. It takes the information from the call site and generates an expression tree representing the operation to which the binder is bound. There is obviously a lot going on here. Great care has been taken to optimize what would appear to be a very complex operation. Clearly, although using the dynamic type can be useful, it does come with a price. Hosting the DLR ScriptRuntime Imagine being able to add scripting capabilities to an application, or passing values in and out of the script so the application can take advantage of the work that the script does. These are the kind of capabilities that hosting the DLR’s ScriptRuntime in your app gives you. Currently, IronPython, IronRuby, and JavaScript are supported as hosted scripting languages. The ScriptRuntime enables you to execute snippets of code or a complete script stored in a file. You can select the proper language engine or allow the DLR to figure out which engine to use. The script can be created in its own app domain or in the current one. Not only can you pass values in and out of the script, you can call methods on dynamic objects created in the script. This degree of flexibility provides countless uses for hosting the ScriptRuntime. The following example demonstrates one way that you can use the ScriptRuntime. Imagine a shopping cart application. One of the requirements is to calculate a discount based on certain criteria. These discounts change often as new sales campaigns are started and completed. There are many ways to handle such a requirement; this example shows how it could be done using the ScriptRuntime and a little Python scripting. For simplicity, the example is a Windows client app. It could be part of a larger web application or any other application. Figure 12-1 shows a sample screen for the application. Figure 12-1 c12.indd 318 30-01-2014 20:17:02 Hosting the DLR ScriptRuntime  ❘  319 Using the values provided for the number of items and the total cost of the items, the application applies a discount based on which radio button is selected. In a real application, the system would use a slightly more sophisticated technique to determine the discount to apply, but for this example the radio buttons will suffice. Here is the code that performs the discount: private void button1_Click(object sender, RoutedEventArgs e) { string scriptToUse; if (CostRadioButton.IsChecked.Value) { scriptToUse = "AmountDisc.py"; } else { scriptToUse = "CountDisc.py"; } ScriptRuntime scriptRuntime = ScriptRuntime.CreateFromConfiguration(); ScriptEngine pythEng = scriptRuntime.GetEngine("Python"); ScriptSource source = pythEng.CreateScriptSourceFromFile(scriptToUse); ScriptScope scope = pythEng.CreateScope(); scope.SetVariable("prodCount", Convert.ToInt32(totalItems.Text)); scope.SetVariable("amt", Convert.ToDecimal(totalAmt.Text)); source.Execute(scope); label5.Content = scope.GetVariable("retAmt").ToString(); } The first part just determines which script to apply, AmountDisc.py or CountDisc.py. AmountDisc.py does the discount based on the amount of the purchase: discAmt = .25 retAmt = amt if amt > 25.00: retAmt = amt-(amt*discAmt) The minimum amount needed for a discount to be applied is $25. If the amount is less than that, then no discount is applied; otherwise, a discount of 25 percent is applied. ContDisc.py applies the discount based on the number of items purchased: discCount = 5 discAmt = .1 retAmt = amt if prodCount > discCount: retAmt = amt-(amt*discAmt) In this Python script, the number of items purchased must be more than 5 for a 10 percent discount to be applied to the total cost. The next step is getting the ScriptRuntime environment set up. For this, four specific tasks are performed: creating the ScriptRuntime object, setting the proper ScriptEngine, creating the ScriptSource, and creating the ScriptScope. The ScriptRuntime object is the starting point, or base, for hosting. It contains the global state of the hosting environment. The ScriptRuntime is created using the CreateFromConfiguration static method. This is what the app.config file looks like:
The code defines a section for “microsoft.scripting” and sets a couple of properties for the IronPython language engine. Next, you get a reference to the ScriptEngine from the ScriptRuntime. In the example, you specify that you want the Python engine, but the ScriptRuntime would have been able to determine this on its own because of the py extension on the script. The ScriptEngine does the work of executing the script code. There are several methods for executing scripts from files or from snippets of code. The ScriptEngine also gives you the ScriptSource and ScriptScope. The ScriptSource object is what gives you access to the script. It represents the source code of the script. With it you can manipulate the source of the script, load it from a disk, parse it line by line, and even compile the script into a CompiledCode object. This is handy if the same script is executed multiple times. The ScriptScope object is essentially a namespace. To pass a value into or out of a script, you bind a variable to the ScriptScope. In the following example, you call the SetVariable method to pass into the Python script the prodCount variable and the amt variable. These are the values from the totalItems text box and the totalAmt text box. The calculated discount is retrieved from the script by using the GetVariable method. In this example, the retAmt variable has the value you’re looking for. The CalcTax button illustrates how to call a method on a Python object. The script CalcTax.py is a very simple method that takes an input value, adds 7.5 percent tax, and returns the new value. Here’s what the code looks like: def CalcTax(amount): return amount*1.075 Here is the C# code to call the CalcTax method: private void button2_Click(object sender, RoutedEventArgs e) { ScriptRuntime scriptRuntime = ScriptRuntime.CreateFromConfiguration(); dynamic calcRate = scriptRuntime.UseFile("CalcTax.py"); label6.Content = calcRate.CalcTax(Convert.ToDecimal(label5.Content)).ToString(); } c12.indd 320 30-01-2014 20:17:02 DynamicObject and ExpandoObject  ❘  321 A very simple process — you create the ScriptRuntime object using the same configuration settings as before. calcRate is a ScriptScope object. You defined it as dynamic so you can easily call the CalcTax method. This is an example of the how the dynamic type can make life a little easier. DynamicObject and ExpandoObject What if you want to create your own dynamic object? You have a couple of options for doing that: by deriving from DynamicObject or by using ExpandoObject. Using DynamicObject is a little more work because you have to override a couple of methods. ExpandoObject is a sealed class that is ready to use. DynamicObject Consider an object that represents a person. Normally, you would define properties for the first name, middle name, and last name. Now imagine the capability to build that object during runtime, with the system having no prior knowledge of what properties the object may have or what methods the object may support. That’s what having a DynamicObject-based object can provide. There may be very few times when you need this sort of functionality, but until now the C# language had no way of accommodating such a requirement. First take a look at what the DynamicObject looks like: class WroxDynamicObject : DynamicObject { Dictionary _dynamicData = new Dictionary(); public override bool TryGetMember(GetMemberBinder binder, out object result) { bool success = false; result = null; if (_dynamicData.ContainsKey(binder.Name)) { result = _dynamicData[binder.Name]; success = true; } else { result = "Property Not Found!"; success = false; } return success; } public override bool TrySetMember(SetMemberBinder binder, object value) { _dynamicData[binder.Name] = value; return true; } public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) { dynamic method = _dynamicData[binder.Name]; result = method((DateTime)args[0]); return result != null; } } c12.indd 321 30-01-2014 20:17:02 322  ❘  CHAPTER 12  Dynamic Language Extensions In this example, you’re overriding three methods: TrySetMember, TryGetMember, and TryInvokeMember. TrySetMember adds the new method, property, or field to the object. In this case, you store the member information in a Dictionary object. The SetMemberBinder object that is passed into the TrySetMember method contains the Name property, which is used to identify the element in the Dictionary. The TryGetMember retrieves the object stored in the Dictionary based on the GetMemberBinder Name property. Here is the code that makes use of the new dynamic object just created: dynamic wroxDyn = new WroxDynamicObject(); wroxDyn.FirstName = "Bugs"; wroxDyn.LastName = "Bunny"; Console.WriteLine(wroxDyn.GetType()); Console.WriteLine("{0} {1}", wroxDyn.FirstName, wroxDyn.LastName); It looks simple enough, but where is the call to the methods you overrode? That’s where the .NET Framework helps. DynamicObject handles the binding for you; all you have to do is reference the properties FirstName and LastName as if they were there all the time. Adding a method is also easily done. You can use the same WroxDynamicObject and add a GetTomorrowDate method to it. It takes a DateTime object and returns a date string representing the next day. Here’s the code: dynamic wroxDyn = new WroxDynamicObject(); Func GetTomorrow = today => today.AddDays(1).ToShortDateString(); wroxDyn.GetTomorrowDate = GetTomorrow; Console.WriteLine("Tomorrow is {0}", wroxDyn.GetTomorrowDate(DateTime.Now)); You create the delegate GetTomorrow using Func. The method the delegate represents is the call to AddDays. One day is added to the Date that is passed in, and a string of that date is returned. The delegate is then set to GetTomorrowDate on the wroxDyn object. The last line calls the new method, passing in the current day’s date. Hence the dynamic magic and you have an object with a valid method. ExpandoObject ExpandoObject works similarly to the WroxDynamicObject created in the previous section. The difference is that you don’t have to override any methods, as shown in the following code example: static void DoExpando() { dynamic expObj = new ExpandoObject(); expObj.FirstName = "Daffy"; expObj.LastName = "Duck"; Console.WriteLine(expObj.FirstName + " " + expObj.LastName); Func GetTomorrow = today => today.AddDays(1).ToShortDateString(); expObj.GetTomorrowDate = GetTomorrow; Console.WriteLine("Tomorrow is {0}", expObj.GetTomorrowDate(DateTime.Now)); expObj.Friends = new List(); expObj.Friends.Add(new Person() { FirstName = "Bob", LastName = "Jones" }); expObj.Friends.Add(new Person() { FirstName = "Robert", LastName = "Jones" }); expObj.Friends.Add(new Person() { FirstName = "Bobby", LastName = "Jones" }); foreach (Person friend in expObj.Friends) { Console.WriteLine(friend.FirstName + " " + friend.LastName); } } c12.indd 322 30-01-2014 20:17:02 DynamicObject and ExpandoObject  ❘  323 Notice that this code is almost identical to what you did earlier. You add a FirstName and LastName property, add a GetTomorrow function, and then do one additional thing — add a collection of Person objects as a property of the object. At first glance it may seem that this is no different from using the dynamic type, but there are a couple of subtle differences that are important. First, you can’t just create an empty dynamic typed object. The dynamic type has to have something assigned to it. For example, the following code won’t work: dynamic dynObj; dynObj.FirstName = "Joe"; As shown in the previous example, this is possible with ExpandoObject. Second, because the dynamic type has to have something assigned to it, it will report back the type assigned to it if you do a GetType call. For example, if you assign an int, it will report back that it is an int. This won’t happen with ExpandoObject or an object derived from DynamicObject. If you have to control the addition and access of properties in your dynamic object, then deriving from DynamicObject is your best option. With DynamicObject, you can use several methods to override and control exactly how the object interacts with the runtime. For other cases, using the dynamic type or the ExpandoObject may be appropriate. Following is another example of using dynamic and ExpandoObject. Assume that the requirement is to develop a general-purpose comma-separated values (CSV) file parsing tool. You won’t know from one execution to another what data will be in the file, only that the values will be comma-separated and that the first line will contain the field names. First, open the file and read in the stream. A simple helper method can be used to do this: private StreamReader OpenFile(string fileName) { if(File.Exists(fileName)) { return new StreamReader(fileName); } return null; } This just opens the file and creates a new StreamReader to read the file contents. Now you want to get the field names. This is easily done by reading in the first line from the file and using the Split function to create a string array of field names: string[] headerLine = fileStream.ReadLine().Split(','); Next is the interesting part. You read in the next line from the file, create a string array just like you did with the field names, and start creating your dynamic objects. Here’s what the code looks like: var retList = new List(); while (fileStream.Peek() > 0) { string[] dataLine = fileStream.ReadLine().Split(','); dynamic dynamicEntity = new ExpandoObject(); for(int i=0;i)dynamicEntity).Add(headerLine[i],dataLine[i]); } retList.Add(dynamicEntity); } c12.indd 323 30-01-2014 20:17:02 324  ❘  CHAPTER 12  Dynamic Language Extensions Once you have the string array of field names and data elements, you create a new ExpandoObject and add the data to it. Notice that you cast the ExpandoObject to a Dictionary object. You use the field name as the key and the data as the value. Then you can add the new object to the retList object you created and return it to the code that called the method. What makes this nice is you have a section of code that can handle any data you give it. The only requirements in this case are ensuring that the field names are the first line and that everything is comma- separated. This concept could be expanded to other file types or even to a DataReader. Summary In this chapter we looked at how the dynamic type can change the way you look at C# programming. Using ExpandoObject in place of multiple objects can reduce the number of lines of code significantly. Also using the DLR and adding scripting languages like Python or Ruby can help building a more polymorphic application that can be changed easily without re-compiling. Dynamic development is becoming increasingly popular because it enables you to do things that are very difficult in a statically typed language. The dynamic type and the DLR enable C# programmers to make use of some dynamic capabilities. c12.indd 324 30-01-2014 20:17:03 Asynchronous Programming WHAT’s in THis CHAPTER? ➤ Why asynchronous programming is important ➤ Asynchronous patterns ➤ Foundations of the async and await keywords ➤ Creating and using asynchronous methods ➤ Error handling with asynchronous methods WRoX.Com CodE doWnloAds FoR THis CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the download Code tab. The code for this chapter is divided into the following major examples: ➤ Async Patterns ➤ Foundations ➤ Error Handling WHy AsynCHRonous PRogRAmming is imPoRTAnT The most important change of C# 5 is the advances provided with asynchronous programming. C# 5 adds only two new keywords: async and await. These two keywords are the main focus of this chapter. With asynchronous programming a method is called that runs in the background (typically with the help of a thread or task), and the calling thread is not blocked. In this chapter, you can read about different patterns on asynchronous programming such as the asynchronous pattern, the event-based asynchronous pattern, and the new task-based asynchronous pattern (TAP). TAP makes use of the async and await keywords. Comparing these patterns you can see the real advantage of the new style of asynchronous programming. After discussing the different patterns, you will see the foundation of asynchronous programming by creating tasks and invoking asynchronous methods. You’ll learn about what’s behind the scenes with continuation tasks and the synchronization context. 13 c13.indd 325 30-01-2014 20:17:39 326  ❘  CHAPTER 13  Asynchronous Programming Error handling needs some special emphasis; as with asynchronous tasks, some scenarios require some dif- ferent handling with errors. The last part of this chapter discusses how cancellation can be done. Background tasks can take a while and there might be a need to cancel the task while it is still running. How this can be done, you’ll also read in this chapter. Chapter 21, “Tasks, Threads, and Synchronization” covers other information about parallel programming. Users find it annoying when an application does not immediately react to requests. With the mouse, we have become accustomed to experiencing a delay, as we’ve learned that behavior over several decades. With a touch UI, an application needs to immediately react to requests. Otherwise, the user tries to redo the action. Because asynchronous programming was hard to achieve with older versions of the .NET Framework, it was not always done when it should have been. One of the applications that blocked the UI thread fairly often is Visual Studio 2010. With that version, opening a solution containing hundreds of projects meant you could take a long coffee break. Since Visual Studio 2012, that’s no longer the case, as projects are loaded asyn- chronously in the background, with the selected project loaded first. This loading behavior is just one exam- ple of important changes built into Visual Studio 2012 related to asynchronous programming. Similarly, users of Visual Studio 2010 are likely familiar with the experience of a dialog not reacting. This is less likely to occur with Visual Studio 2012 and 2013. Many APIs with the .NET Framework offer both a synchronous and an asynchronous version. Because the synchronous version of the API was a lot easier to use, it was often used where it wasn’t appropriate. With the new Windows Runtime (WinRT), if an API call is expected to take longer than 40 milliseconds, only an asynchronous version is available. Now, with .NET 4.5 programming, asynchronously is as easy as pro- gramming in a synchronous manner, so there shouldn’t be any barrier to using the asynchronous APIs. Asynchronous Patterns Before stepping into the new async and await keywords it is best to understand asynchronous patterns from the .NET Framework. Asynchronous features have been available since .NET 1.0, and many classes in the .NET Framework implement one or more such patterns. The asynchronous pattern is also available with the delegate type. Because doing updates on the UI, both with Windows Forms, and WPF with the asynchronous pattern is quite complex, .NET 2.0 introduced the event-based asynchronous pattern. With this pattern, an event handler is invoked from the thread that owns the synchronization context, so updating UI code is easily handled with this pattern. Previously, this pattern was also known with the name asynchronous component pattern. Now, with .NET 4.5, another new way to achieve asynchronous programming is introduced: the task-based asynchronous pattern (TAP). This pattern is based on the Task type that was new with .NET 4 and makes use of a compiler feature with the keywords async and await. To understand the advantage of the async and await keywords, the first sample application makes use of Windows Presentation Foundation (WPF) and network programming to provide an overview of asynchro- nous programming. If you have no experience with WPF and network programming, don’t despair. You can still follow the essentials here and gain an understanding of how asynchronous programming can be done. The following examples demonstrate the differences between the asynchronous patterns. After looking at these, you’ll learn the basics of asynchronous programming with some simple console applications. Note  WPF is covered in detail in Chapters 35, “Core WPF,” and 36, “Business Applications with WPF,” and network programming is discussed in Chapter 26, “Networking.” c13.indd 326 30-01-2014 20:17:39 Asynchronous Patterns  ❘  327 The sample application to show the differences between the asynchronous patterns is a WPF application that makes use of types in a class library. The application is used to find images on the web using services from Bing and Flickr. The user can enter a search term to find images, and the search term is sent to Bing and Flickr services with a simple HTTP request. The UI design from the Visual Studio designer is shown in Figure 13-1. On top of the screen is a text input field followed by several buttons that start the search or clear the result list. The left side below the control area contains a ListBox for displaying all the images found. On the right side is an Image control to display the image that is selected within the ListBox control in a version with a higher resolution. Figure 13-1 To understand the sample application we will start with the class library AsyncLib, which contains several helper classes. These classes are used by the WPF application. The class SearchItemResult represents a single item from a result collection that is used to display the image together with a title and the source of the image. This class just defines simple properties: Title, Url, ThumbnailUrl, and Source. The property ThumbnailIUrl is used to reference a thumbnail image, the Url property contains a link to a larger-size image. Title contains some text to describe the image. The base class of SearchItemResult is BindableBase. This base class just implements a notification mechanism by implementing the interface INotifyPropertyChanged that is used by WPF to make updates with data bind- ing (code file AsyncLib/SearchItemResult.cs): namespace Wrox.ProCSharp.Async { public class SearchItemResult : BindableBase { private string title; public string Title { get { return title; } set { SetProperty(ref title, value); } } private string url; public string Url { get { return url; } set { SetProperty(ref url, value); } } c13.indd 327 30-01-2014 20:17:39 328  ❘  CHAPTER 13  Asynchronous Programming private string thumbnailUrl; public string ThumbnailUrl { get { return thumbnailUrl; } set { SetProperty(ref thumbnailUrl, value); } } private string source; public string Source { get { return source; } set { SetProperty(ref source, value); } } } } The class SearchInfo is another class used with data binding. The property SearchTerm contains the user input to search for images with that type. The List property returns a list of all found images represented with the SearchItemResult type (code file AsyncLib/SearchInfo.cs): using System.Collections.ObjectModel; namespace Wrox.ProCSharp.Async { public class SearchInfo : BindableBase { public SearchInfo() { list = new ObservableCollection(); list.CollectionChanged += delegate { OnPropertyChanged("List"); }; } private string searchTerm; public string SearchTerm { get { return searchTerm; } set { SetProperty(ref searchTerm, value); } } private ObservableCollection list; public ObservableCollection List { get { return list; } } } } In the XAML code, a TextBox is used to enter the search term. This control is bound to the SearchTerm property of the SearchInfo type. Several Button controls are used to activate an event handler, e.g., the Sync button invokes the OnSearchSync method (XAML file AsyncPatterns/MainWindow.xaml): c13.indd 328 30-01-2014 20:17:39 Asynchronous Patterns  ❘  329 The second part of the XAML code contains a ListBox. To have a special representation for the items in the ListBox, an ItemTemplate is used. Every item is represented with two TextBlock controls and one Image control. The ListBox is bound to the List property of the SearchInfo class, and properties of the item controls are bound to properties of the SearchItemResult type: Now let’s get into the BingRequest class. This class contains some information about how to make a request to the Bing service. The Url property of this class returns a URL string that can be used to make a request for images. The request is comprised of the search term, a number of images that should be requested (Count), and a number of images to skip (Offset). With Bing, authentication is needed. The user id is defined with the AppId, and used with the Credentials property that returns a NetworkCredential object. To run the application, you need to register with Windows Azure Marketplace and sign up for the Bing Search API. At the time of this writing, up to 5000 transactions per month are free—this should be enough for running the sample application. Every search is one transaction. The link for the registration to the Bing Search API is https://datamarket.azure.com/dataset/bing/search. After registration you need to copy the application id. After obtaining the application ID, add it to the BingRequest class. After sending a request to Bing by using the created URL, Bing returns XML. The Parse method of the BingRequest class parses the XML and returns a collection of SearchItemResult objects (code file AsyncLib/BingRequest.cs): Note  The Parse methods in the classes BingRequest and FlickrRequest make use of LINQ to XML. How to use LINQ to XML is covered in Chapter 34, “Manipulating XML.” c13.indd 329 30-01-2014 20:17:39 330  ❘  CHAPTER 13  Asynchronous Programming using System.Collections.Generic; using System.Linq; using System.Net; using System.Xml.Linq; namespace Wrox.ProCSharp.Async { public class BingRequest : IImageRequest { private const string AppId = "enter your Bing AppId here"; public BingRequest() { Count = 50; Offset = 0; } private string searchTerm; public string SearchTerm { get { return searchTerm; } set { searchTerm = value; } } public ICredentials Credentials { get { return new NetworkCredentials(AppId, AppId); } } public string Url { get { return string.Format("https://api.datamarket.azure.com/" + "Data.ashx/Bing/Search/v1/Image?Query=%27{0}%27&" + "$top={1}&$skip={2}&$format=Atom", SearchTerm, Count, Offset); } } public int Count { get; set; } public int Offset { get; set; } public IEnumerable Parse(string xml) { XElement respXml = XElement.Parse(xml); // XNamespace atom = XNamespace.Get("http://www.w3.org/2005/Atom"); XNamespace d = XNamespace.Get( "http://schemas.microsoft.com/ado/2007/08/dataservices"); XNamespace m = XNamespace.Get( "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"); return (from item in respXml.Descendants(m + "properties") select new SearchItemResult { Title = new string(item.Element(d + "Title").Value.Take(50).ToArray()), Url = item.Element(d + "MediaUrl").Value, ThumbnailUrl = item.Element(d + "Thumbnail"). Element(d + "MediaUrl").Value, c13.indd 330 30-01-2014 20:17:39 Asynchronous Patterns  ❘  331 Source = "Bing" }).ToList(); } } } Both the BingRequest class and the FlickrRequest class implement the interface IImageRequest. This interface defines the properties SearchTerm and Url, and the method Parse, which enables easy iteration through both image service providers (code file AsyncLib/IImageRequest.cs): using System; using System.Collections.Generic; using System.Net; namespace Wrox.ProCSharp.Async { public interface IImageRequest { string SearchTerm { get; set; } string Url { get; } IEnumerable Parse(string xml); ICredentials Credentials { get; } } } The FlickrRequest class is very similar to BingRequest. It just creates a different URL to request an image with a search term, and has a different implementation of the Parse method, just as the returned XML from Flickr differs from the returned XML from Bing. As with Bing, to create an application ID for Flickr, you need to register with Flickr and request it: http://www.flickr.com/services/apps/create/ apply/. using System.Collections.Generic; using System.Linq; using System.Xml.Linq; namespace Wrox.ProCSharp.Async { public class FlickrRequest : IImageRequest { private const string AppId = "Enter your Flickr AppId here"; public FlickrRequest() { Count = 50; Page = 1; } private string searchTerm; public string SearchTerm { get { return searchTerm; } set { searchTerm = value; } } public string Url { get c13.indd 331 30-01-2014 20:17:40 332  ❘  CHAPTER 13  Asynchronous Programming { return string.Format("http://api.flickr.com/services/rest?" + "api_key={0}&method=flickr.photos.search&content_type=1&" + "text={1}&per_page={2}&page={3}", AppId, SearchTerm, Count, Page); } } public ICredentials Credentials { get { return null; } } public int Count { get; set; } public int Page { get; set; } public IEnumerable Parse(string xml) { XElement respXml = XElement.Parse(xml); return (from item in respXml.Descendants("photo") select new SearchItemResult { Title = new string(item.Attribute("title").Value. Take(50).ToArray()), Url = string.Format("http://farm{0}.staticflickr.com/" + "{1}/{2}_{3}_z.jpg", item.Attribute("farm").Value, item.Attribute("server").Value, item.Attribute("id").Value, item.Attribute("secret").Value), ThumbnailUrl = string.Format("http://farm{0}." + "staticflickr.com/{1}/{2}_{3}_t.jpg", item.Attribute("farm").Value, item.Attribute("server").Value, item.Attribute("id").Value, item.Attribute("secret").Value), Source = "Flickr" }).ToList(); } } } Now you just need to connect the types from the library and the WPF application. In the constructor of the MainWindow class, an instance of SearchInfo is created, and the DataContext of the window is set to this instance. Now data binding can take place, shown earlier with the XAML code (code file AsyncPatterns/ MainWindow.xaml.cs): public partial class MainWindow : Window { private SearchInfo searchInfo; public MainWindow() { InitializeComponent(); searchInfo = new SearchInfo(); this.DataContext = searchInfo; } The MainWindow class also contains the helper method GetSearchRequests, which returns a collection of IImageRequest objects in the form of BingRequest and FlickrRequest types. In case you only registered with one of these services, you can change this code to return only the one with which you registered. Of course, you can also create IImageRequest types of other services, e.g., using Google or Yahoo. Then add these request types to the collection returned: c13.indd 332 30-01-2014 20:17:40 Asynchronous Patterns  ❘  333 private IEnumerable GetSearchRequests() { return new List { new BingRequest { SearchTerm = searchInfo.SearchTerm }, new FlickrRequest { SearchTerm = searchInfo.SearchTerm} }; } Synchronous Call Now that everything is set up, let’s start with a synchronous call to these services. The click handler of the Sync button, OnSearchSync, iterates through all search requests returned from GetSearchRequests and uses the Url property to make an HTTP request with the WebClient class. The method DownloadString blocks until the result is received. The resulting XML is assigned to the resp variable. The XML con- tent is parsed with the help of the Parse method, which returns a collection of SearchItemResult objects. The items of these collections are then added to the list contained within searchInfo (code file AsyncPatterns/MainWindow.xaml.cs): private void OnSearchSync(object sender, RoutedEventArgs e) { foreach (var req in GetSearchRequests()) { var client = new WebClient(); client.Credentials = req.Credentials; string resp = client.DownloadString(req.Url); IEnumerable images = req.Parse(resp); foreach (var image in images) { searchInfo.List.Add(image); } } } Running the application (see Figure 13-2), the user interface is blocked until the method OnSearchSync is finished making network calls to Bing and Flickr, as well as parsing the results. The amount of time needed to complete these calls varies according to the speed of your network and the current workload of Bing and Flickr. Whatever it is, however, the wait is unpleasant to the user. Figure 13-2 Therefore, make the call asynchronously instead. c13.indd 333 30-01-2014 20:17:40 334  ❘  CHAPTER 13  Asynchronous Programming Asynchronous Pattern One way to make the call asynchronously is by using the asynchronous pattern. The asynchro- nous pattern defines a BeginXXX method and an EndXXX method. For example, if a synchronous method DownloadString is offered, the asynchronous variants would be BeginDownloadString and EndDownloadString. The BeginXXX method takes all input arguments of the synchronous method, and EndXXX takes the output arguments and return type to return the result. With the asynchronous pattern, the BeginXXX method also defines a parameter of AsyncCallback, which accepts a delegate that is invoked as soon as the asynchronous method is completed. The BeginXXX method returns IAsyncResult, which can be used for polling to verify whether the call is completed, and to wait for the end of the method. The WebClient class doesn’t offer an implementation of the asynchronous pattern. Instead, the HttpWebRequest class could be used, which offers this pattern with the methods BeginGetResponse and EndGetResponse. This is not done in the following sample. Instead, a delegate is used. The delegate type defines an Invoke method to make a synchronous method call, and BeginInvoke and EndInvoke meth- ods to use it with the asynchronous pattern. Here, the delegate downloadString of type Func is declared to reference a method that has a string parameter and returns a string. The method that is referenced by the downloadString variable is implemented as a Lambda expression and invokes the synchronous method DownloadString of the WebClient type. The delegate is invoked asynchronously by calling the BeginInvoke method. This method uses a thread from the thread pool to make an asynchronous call. The first parameter of the BeginInvoke method is the first generic string parameter of the Func delegate where the URL can be passed. The second parameter is of type AsyncCallback. AsyncCallback is a del- egate that requires IAsyncResult as a parameter. The method referenced by this delegate is invoked as soon as the asynchronous method is completed. When that happens, downloadString.EndInvoke is invoked to retrieve the result, which is dealt with in the same manner as before to parse the XML content and get the collection of items. However, here it is not possible to directly go back to the UI, as the UI is bound to a single thread, and the callback method is running within a background thread. Therefore, it’s necessary to switch back to the UI thread by using the Dispatcher property from the window. The Invoke method of the Dispatcher requires a delegate as a parameter; that’s why the Action delegate is specified, which adds an item to the collection bound to the UI (code file AsyncPatterns/MainWindow .xaml.cs): private void OnSeachAsyncPattern(object sender, RoutedEventArgs e) { Func downloadString = (address, cred) => { var client = new WebClient(); client.Credentials = cred; return client.DownloadString(address); }; Action addItem = item => searchInfo.List.Add(item); foreach (var req in GetSearchRequests()) { downloadString.BeginInvoke(req.Url, req.Credentials, ar => { string resp = downloadString.EndInvoke(ar); IEnumerable images = req.Parse(resp); foreach (var image in images) { this.Dispatcher.Invoke(addItem, image); } }, null); } } c13.indd 334 30-01-2014 20:17:40 Asynchronous Patterns  ❘  335 An advantage of the asynchronous pattern is that it can be implemented easily just by using the functional- ity of delegates. The program now behaves as it should; the UI is no longer blocked. However, using the asynchronous pattern is difficult. Fortunately, .NET 2.0 introduced the event-based asynchronous pattern, which makes it easier to deal with UI updates. This pattern is discussed next. Note  Delegate types and lambda expressions are explained in Chapter 8, “Delegates, Lambdas, and Events.” Threads and thread pools are covered in Chapter 21. Event-Based Asynchronous Pattern The method OnAsyncEventPattern makes use of the event-based asynchronous pattern. This pattern is implemented by the WebClient class and thus it can be directly used. This pattern defines a method with the suffix "Async". Therefore, for example, for the synchronous method DownloadString, the WebClient class offers the asynchronous variant DownloadStringAsync. Instead of defining a delegate that is invoked when the asynchronous method is completed, an event is defined. The DownloadStringCompleted event is invoked as soon as the asynchronous method DownloadStringAsync is completed. The method assigned to the event handler is implemented within a lambda expression. The implementation is very similar to before, but now it is possible to directly access UI elements because the event handler is invoked from the thread that has the synchronization context, and this is the UI thread in the case of Windows Forms and WPF applications (code file AsyncPatterns/MainWindow.xaml.cs): private void OnAsyncEventPattern(object sender, RoutedEventArgs e) { foreach (var req in GetSearchRequests()) { var client = new WebClient(); client.Credentials = req.Credentials; client.DownloadStringCompleted += (sender1, e1) => { string resp = e1.Result; IEnumerable images = req.Parse(resp); foreach (var image in images) { searchInfo.List.Add(image); } }; client.DownloadStringAsync(new Uri(req.Url)); } } An advantage of the event-based asynchronous pattern is that it is easy to use. Note, however, that it is not that easy to implement this pattern in a custom class. One way to use an existing implementation of this pat- tern to make synchronous methods asynchronous is with the BackgroundWorker class. BackgroundWorker implements the event-based asynchronous pattern. This makes the code a lot simpler. However, the order is reversed compared to synchronous method calls. Before invoking the asynchronous method, you need to define what happens when the method call is com- pleted. The following section plunges into the new world of asynchronous programming with the async and await keywords. c13.indd 335 30-01-2014 20:17:40 336  ❘  CHAPTER 13  Asynchronous Programming Task-Based Asynchronous Pattern The WebClient class is updated with .NET 4.5 to offer the task-based asynchronous pattern (TAP) as well. This pattern defines a suffix Async method that returns a Task type. Because the WebClient class already offers a method with the Async suffix to implement the task-based asynchronous pattern, the new method has the name DownloadStringTaskAsync. The method DownloadStringTaskAsync is declared to return Task. You do not need to declare a variable of Task to assign the result from DownloadStringTaskAsync; instead, a variable of type string can be declared, and the await keyword used. The await keyword unblocks the thread (in this case the UI thread) to do other tasks. As soon as the method DownloadStringTaskAsync completes its background processing, the UI thread can continue and get the result from the background task to the string variable resp. Also, the code following this line continues (code file AsyncPatterns/MainWindow.xaml.cs): private async void OnTaskBasedAsyncPattern(object sender, RoutedEventArgs e) { foreach (var req in GetSearchRequests()) { var client = new WebClient(); client.Credentials = req.Credentials; string resp = await client.DownloadStringTaskAsync(req.Url); IEnumerable images = req.Parse(resp); foreach (var image in images) { searchInfo.List.Add(image); } } } Note  The async keyword creates a state machine similar to the yield return state- ment, which is discussed in Chapter 6, “Arrays and Tuples.” The code is much simpler now. There is no blocking, and no manually switching back to the UI thread, as this is done automatically; and the code has the same order as you’re used to with synchronous programming. Next, the code is changed to use a different class from WebClient, one in which the task-based event pat- tern is more directly implemented and synchronous methods are not offered. This class, new with .NET 4.5, is HttpClient. Doing an asynchronous GET request is done with the GetAsync method. Then, to read the content another asynchronous method is needed. ReadAsStringAsync returns the content formatted in a string: private async void OnTaskBasedAsyncPattern(object sender, RoutedEventArgs e) { foreach (var req in GetSearchRequests()) { var clientHandler = new HttpClientHandler { Credentials = req.Credentials }; var client = new HttpClient(clientHandler); c13.indd 336 30-01-2014 20:17:41 Asynchronous Patterns  ❘  337 var response = await client.GetAsync(req.Url); string resp = await response.Content.ReadAsStringAsync(); IEnumerable images = req.Parse(resp); foreach (var image in images) { searchInfo.List.Add(image); } } } Parsing of the XML string to could take a while. Because the parsing code is running in the UI thread, the UI thread cannot react to user requests at that time. To create a background task from synchronous functionality, Task.Run can be used. In the following example, Task.Run wraps the parsing of the XML string to return the SearchItemResult collection: private async void OnTaskBasedAsyncPattern(object sender, RoutedEventArgs e) { foreach (var req in GetSearchRequests()) { var clientHandler = new HttpClientHandler { Credentials = req.Credentials }; var client = new HttpClient(clientHandler); var response = await client.GetAsync(req.Url, cts.Token); string resp = await response.Content.ReadAsStringAsync(); await Task.Run(() => { IEnumerable images = req.Parse(resp); foreach (var image in images) { searchInfo.List.Add(image); } } } } Because the method passed to the Task.Run method is running in a background thread, here we have the same problem as before referencing some UI code. One solution would be to just do req.Parse within the Task.Run method, and do the foreach loop outside of the task to add the result to the list in the UI thread. WPF with .NET 4.5 offers a better solution, however, that enables filling collections that are bound to the UI from a background thread. This extension only requires enabling the collection for synchronization using BindingOperations.EnableCollectionSynchronization, as shown in the following code snippet: public partial class MainWindow : Window { private SearchInfo searchInfo; private object lockList = new object(); public MainWindow() { InitializeComponent(); searchInfo = new SearchInfo(); this.DataContext = searchInfo; BindingOperations.EnableCollectionSynchronization( searchInfo.List, lockList); } c13.indd 337 30-01-2014 20:17:41 338  ❘  CHAPTER 13  Asynchronous Programming Having looked at the advantages of the async and await keywords, the next section examines the program- ming foundation behind these keywords. Foundation of Asynchronous Programming The async and await keywords are just a compiler feature. The compiler creates code by using the Task class. Instead of using the new keywords, you could get the same functionality with C# 4 and methods of the Task class; it’s just not as convenient. This section gives information about what the compiler does with the async and await keywords, an easy way to create an asynchronous method, how you can invoke multiple asynchronous methods in parallel, and how you can change a class that just offers the asynchronous pattern to use the new keywords. Creating Tasks Let’s start with the synchronous method Greeting, which takes a while before returning a string (code file Foundations/Program.cs): static string Greeting(string name) { Thread.Sleep(3000); return string.Format("Hello, {0}", name); } To make such a method asynchronously, the method GreetingAsync is defined. The task-based asyn- chronous pattern specifies that an asynchronous method is named with the Async suffix and returns a task. GreetingAsync is defined to have the same input parameters as the Greeting method but returns Task. Task, which defines a task that returns a string in the future. A simple way to return a task is by using the Task.Run method. The generic version Task.Run() creates a task that returns a string: static Task GreetingAsync(string name) { return Task.Run(() => { return Greeting(name); }); } Calling an Asynchronous Method You can call this asynchronous method GreetingAsync by using the await keyword on the task that is returned. The await keyword requires the method to be declared with the async modifier. The code within this method does not continue before the GreetingAsync method is completed. However, the thread that started the CallerWithAsync method can be reused. This thread is not blocked: private async static void CallerWithAsync() { string result = await GreetingAsync("Stephanie"); Console.WriteLine(result); } Instead of passing the result from the asynchronous method to a variable, you can also use the await key- word directly within parameters. Here, the result from the GreetingAsync method is awaited like in the previously code snippet, but this time the result is directly passed to the Console.WriteLine method: c13.indd 338 30-01-2014 20:17:41 Foundation of Asynchronous Programming  ❘  339 private async static void CallerWithAsync2() { Console.WriteLine(await GreetingAsync("Stephanie")); } Note  The async modifier can only be used with methods returning a Task or void. It cannot be used with the entry point of a program, the Main method. await can only be used with methods returning a Task. In the next section you’ll see what’s driving this await keyword. Behind the scenes, continuation tasks are used. Continuation with Tasks GreetingAsync returns a Task object. The Task object contains information about the task cre- ated, and allows waiting for its completion. The ContinueWith method of the Task class defines the code that should be invoked as soon as the task is finished. The delegate assigned to the ContinueWith method receives the completed task with its argument, which allows accessing the result from the task using the Result property: private static void CallerWithContinuationTask() { Task t1 = GreetingAsync("Stephanie"); t1.ContinueWith(t => { string result = t.Result; Console.WriteLine(result); }); } The compiler converts the await keyword by putting all the code that follows within the block of a ContinueWith method. Synchronization Context If you verify the thread that is used within the methods you will find that in both methods, CallerWithAsync and CallerWithContinuationTask, different threads are used during the lifetime of the methods. One thread is used to invoke the method GreetingAsync, and another thread takes action after the await keyword or within the code block in the ContinueWith method. With a console application usually this is not an issue. However, you have to ensure that at least one fore- ground thread is still running before all background tasks that should be completed are finished. The sample application invokes Console.ReadLine to keep the main thread running until the return key is pressed. With applications that are bound to a specific thread for some actions (e.g., with WPF applications, UI ele- ments can only be accessed from the UI thread), this is an issue. Using the async and await keywords you don’t have to do any special actions to access the UI thread after an await completion. By default the generated code switches the thread to the thread that has the synchro- nization context. A WPF application sets a DispatcherSynchronizationContext, and a Windows Forms application sets a WindowsFormsSynchronizationContext. If the calling thread of the asynchronous method is assigned to the synchronization context, then with the continuous execution after the await, by default the same synchronization context is used. If the same synchronization context shouldn’t be used, you c13.indd 339 30-01-2014 20:17:41 340  ❘  CHAPTER 13  Asynchronous Programming must invoke the Task method ConfigureAwait(continueOnCapturedContext: false). An example that illustrates this usefulness is a WPF application in which the code that follows the await is not using any UI elements. In this case, it is faster to avoid the switch to the synchronization context. Using Multiple Asynchronous Methods Within an asynchronous method you can call not only one but multiple asynchronous methods. How you code this depends on whether the results from one asynchronous method are needed by another. Calling Asynchronous Methods Sequentially The await keyword can be used to call every asynchronous method. In cases where one method is dependent on the result of another method, this is very useful. Here, the second call to GreetingAsync is completely independent of the result of the first call to GreetingAsync. Thus, the complete method MultipleAsyncMethods could return the result faster if await is not used with every single method, as shown in the following example: private async static void MultipleAsyncMethods() { string s1 = await GreetingAsync("Stephanie"); string s2 = await GreetingAsync("Matthias"); Console.WriteLine("Finished both methods.\n " + "Result 1: {0}\n Result 2: {1}", s1, s2); } Using Combinators If the asynchronous methods are not dependent on each other, it is a lot faster not to await on each sepa- rately, and instead assign the return of the asynchronous method to a Task variable. The GreetingAsync method returns Task. Both these methods can now run in parallel. Combinators can help with this. A combinator accepts multiple parameters of the same type and returns a value of the same type. The passed parameters are “combined” to one. Task combinators accept multiple Task objects as parameter and return a Task. The sample code invokes the Task.WhenAll combinator method that you can await to have both tasks finished: private async static void MultipleAsyncMethodsWithCombinators1() { Task t1 = GreetingAsync("Stephanie"); Task t2 = GreetingAsync("Matthias"); await Task.WhenAll(t1, t2); Console.WriteLine("Finished both methods.\n " + "Result 1: {0}\n Result 2: {1}", t1.Result, t2.Result); } The Task class defines the WhenAll and WhenAny combinators. The Task returned from the WhenAll method is completed as soon as all tasks passed to the method are completed; the Task returned from the WhenAny method is completed as soon as one of the tasks passed to the method is completed. The WhenAll method of the Task type defines several overloads. If all the tasks return the same type, an array of this type can be used for the result of the await. The GreetingAsync method returns a Task, and awaiting for this method results in a string. Therefore, Task.WhenAll can be used to return a string array: private async static void MultipleAsyncMethodsWithCombinators2() { Task t1 = GreetingAsync("Stephanie"); c13.indd 340 30-01-2014 20:17:41 Error Handling  ❘  341 Task t2 = GreetingAsync("Matthias"); string[] result = await Task.WhenAll(t1, t2); Console.WriteLine("Finished both methods.\n " + "Result 1: {0}\n Result 2: {1}", result[0], result[1]); } Converting the Asynchronous Pattern Not all classes from the .NET Framework introduced the new asynchronous method style with .NET 4.5. There are still many classes just offering the asynchronous pattern with BeginXXX and EndXXX methods and not task-based asynchronous methods as you will see when working with different classes from the framework. First, let’s create an asynchronous method from the previously-defined synchronous method Greeting with the help of a delegate. The Greeting method receives a string as parameter and returns a string, thus a variable of Func delegate is used to reference this method. According to the asyn- chronous pattern, the BeginGreeting method receives a string parameter in addition to AsyncCallback and object parameters and returns IAsyncResult. The EndGreeting method returns the result from the Greeting method—a string—and receives an IAsyncResult parameter. With the implementation just the delegate is used to make the implementation asynchronously. private static Func greetingInvoker = Greeting; static IAsyncResult BeginGreeting(string name, AsyncCallback callback, object state) { return greetingInvoker.BeginInvoke(name, callback, state); } static string EndGreeting(IAsyncResult ar) { return greetingInvoker.EndInvoke(ar); } Now the BeginGreeting and EndGreeting methods are available, and these should be converted to use the async and await keywords to get the results. The TaskFactory class defines the FromAsync method that allows converting methods using the asynchronous pattern to the TAP. With the sample code, the first generic parameter of the Task type, Task, defines the return value from the method that is invoked. The generic parameter of the FromAsync method defines the input type of the method. In this case the input type is again of type string. With the parameters of the FromAsync method, the first two parameters are delegate types to pass the addresses of the BeginGreeting and EndGreeting methods. After these two parameters, the input parameters and the object state parameter follow. The object state is not used, so null is assigned to it. Because the FromAsync method returns a Task type, in the sample code Task, an await can be used as shown: private static async void ConvertingAsyncPattern() { string s = await Task.Factory.FromAsync( BeginGreeting, EndGreeting, "Angela", null); Console.WriteLine(s); } Error Handling Chapter 16, “Errors and Exceptions,” provides detailed coverage of errors and exception handling. However, in the context of asynchronous methods, you should be aware of some special handling of errors. c13.indd 341 30-01-2014 20:17:41 342  ❘  CHAPTER 13  Asynchronous Programming Let’s start with a simple method that throws an exception after a delay (code file ErrorHandling/Program.cs): static async Task ThrowAfter(int ms, string message) { await Task.Delay(ms); throw new Exception(message); } If you call the asynchronous method without awaiting it, you can put the asynchronous method within a try/catch block—and the exception will not be caught. That’s because the method DontHandle has already completed before the exception from ThrowAfter is thrown. You need to await the ThrowAfter method, as shown in the following example: private static void DontHandle() { try { ThrowAfter(200, "first"); // exception is not caught because this method is finished // before the exception is thrown } catch (Exception ex) { Console.WriteLine(ex.Message); } } Warning  Asynchronous methods that return void cannot be awaited. The issue with this is that exceptions that are thrown from async void methods cannot be caught. That’s why it is best to return a Task type from an asynchronous method. Handler methods or overridden base methods are exempted from this rule. Handling Exceptions with Asynchronous Methods A good way to deal with exceptions from asynchronous methods is to use await and put a try/catch state- ment around it, as shown in the following code snippet. The HandleOneError method releases the thread after calling the ThrowAfter method asynchronously, but it keeps the Task referenced to continue as soon as the task is completed. When that happens (which in this case is when the exception is thrown after two seconds), the catch matches and the code within the catch block is invoked: private static async void HandleOneError() { try { await ThrowAfter(2000, "first"); } catch (Exception ex) { Console.WriteLine("handled {0}", ex.Message); } } c13.indd 342 30-01-2014 20:17:42 Error Handling  ❘  343 Exceptions with Multiple Asynchronous Methods What if two asynchronous methods are invoked that each throw exceptions? In the following example, first the ThrowAfter method is invoked, which throws an exception with the message first after two seconds. After this method is completed, the ThrowAfter method is invoked, throwing an exception after one second. Because the first call to ThrowAfter already throws an exception, the code within the try block does not continue to invoke the second method, instead landing within the catch block to deal with the first exception: private static async void StartTwoTasks() { try { await ThrowAfter(2000, "first"); await ThrowAfter(1000, "second"); // the second call is not invoked // because the first method throws // an exception } catch (Exception ex) { Console.WriteLine("handled {0}", ex.Message); } } Now let’s start the two calls to ThrowAfter in parallel. The first method throws an exception after two seconds, the second one after one second. With Task.WhenAll you wait until both tasks are completed, whether an exception is thrown or not. Therefore, after a wait of about two seconds, Task.WhenAll is completed, and the exception is caught with the catch statement. However, you will only see the exception information from the first task that is passed to the WhenAll method. It’s not the task that threw the excep- tion first (which is the second task), but the first task in the list: private async static void StartTwoTasksParallel() { try { Task t1 = ThrowAfter(2000, "first"); Task t2 = ThrowAfter(1000, "second"); await Task.WhenAll(t1, t2); } catch (Exception ex) { // just display the exception information of the first task // that is awaited within WhenAll Console.WriteLine("handled {0}", ex.Message); } } One way to get the exception information from all tasks is to declare the task variables t1 and t2 outside of the try block, so they can be accessed from within the catch block. Here you can check the status of the task to determine whether they are in a faulted state with the IsFaulted property. In case of an exception, the IsFaulted property returns true. The exception information itself can be accessed by using Exception.InnerException of the Task class. Another, and usually better, way to retrieve exception information from all tasks is demonstrated next. Using AggregateException Information To get the exception information from all failing tasks, the result from Task.WhenAll can be written to a Task variable. This task is then awaited until all tasks are completed. Otherwise the exception would still be c13.indd 343 30-01-2014 20:17:42 344  ❘  CHAPTER 13  Asynchronous Programming missed. As described in the last section, with the catch statement just the exception of the first task can be retrieved. However, now you have access to the Exception property of the outer task. The Exception property is of type AggregateException. This exception type defines the property InnerExceptions (not only InnerException), which contains a list of all the exceptions from the awaited for. Now you can easily iterate through all the exceptions: private static async void ShowAggregatedException() { Task taskResult = null; try { Task t1 = ThrowAfter(2000, "first"); Task t2 = ThrowAfter(1000, "second"); await (taskResult = Task.WhenAll(t1, t2)); } catch (Exception ex) { Console.WriteLine("handled {0}", ex.Message); foreach (var ex1 in taskResult.Exception.InnerExceptions) { Console.WriteLine("inner exception {0}", ex1.Message); } } } Cancellation With background tasks that can run longer in some scenarios, it is useful to cancel the tasks. For cancella- tion, .NET offers a standard mechanism that has been available since .NET 4. This mechanism can be used with the task-based asynchronous pattern. The cancellation framework is based on cooperative behavior; it is not forceful. A long-running task needs to check itself if it is canceled, in which case it is the responsibility of the task to cleanup any open resources and finish its work. Cancellation is based on the CancellationTokenSource class, which can be used to send cancel requests. Requests are sent to tasks that reference the CancellationToken that is associated with the CancellationTokenSource. The following section looks at an example by modifying the AsyncPatterns sample created earlier in this chapter to add support for cancellation. Starting a Cancellation First, a variable cts of type CancellationTokenSource is defined with the private field members of the class MainWindow. This member will be used to cancel tasks and pass tokens to the methods that should be cancelled (code file AsyncPatterns/MainWindow.xaml.cs): public partial class MainWindow : Window { private SearchInfo searchInfo; private object lockList = new object(); private CancellationTokenSource cts; For a new button that can be activated by the user to cancel the running task, the event handler method OnCancel is added. Within this method, the variable cts is used to cancel the tasks with the Cancel method: c13.indd 344 30-01-2014 20:17:42 Cancellation  ❘  345 private void OnCancel(object sender, RoutedEventArgs e) { if (cts != null) cts.Cancel(); } The CancellationTokenSource also supports cancellation after a specified amount of time. The method CancelAfter enables passing a value, in milliseconds, after which a task should be cancelled. Cancellation with Framework Features Now let’s pass the CancellationToken to an asynchronous method. Several of the asynchronous meth- ods in the framework support cancellation by offering an overload whereby a CancellationToken can be passed. One example is the GetAsync method of the HttpClient class. The overloaded GetAsync method accepts a CancellationToken in addition to the URI string. The token from the CancellationTokenSource can be retrieved by using the Token property. The implementation of the GetAsync method periodically checks whether the operation should be cancelled. If so, it does a cleanup of resources before throwing the exception OperationCanceledException. This exception is caught with the catch handler in the following code snippet: private async void OnTaskBasedAsyncPattern(object sender, RoutedEventArgs e) { cts = new CancellationTokenSource(); try { foreach (var req in GetSearchRequests()) { var client = new HttpClient(); var response = await client.GetAsync(req.Url, cts.Token); string resp = await response.Content.ReadAsStringAsync(); //... } } catch (OperationCanceledException ex) { MessageBox.Show(ex.Message); } } Cancellation with Custom Tasks What about custom tasks that should be cancelled? The Run method of the Task class offers an overload to pass a CancellationToken as well. However, with custom tasks it is necessary to check whether cancella- tion is requested. In the following example, this is implemented within the foreach loop. The token can be checked by using the IsCancellationRequsted property. If you need to do some cleanup before throwing the exception, it is best to verify that cancellation is requested. If cleanup is not needed, an exception can be fired immediately after the check, which is done with the ThrowIfCancellationRequested method: await Task.Run(() => { var images = req.Parse(resp); foreach (var image in images) c13.indd 345 30-01-2014 20:17:42 346  ❘  CHAPTER 13  Asynchronous Programming { cts.Token.ThrowIfCancellationRequested(); searchInfo.List.Add(image); } }, cts.Token); Now the user can cancel long-running tasks. Summary This chapter introduced the async and await keywords that are new with C# 5. Having looked at several examples, you’ve seen the advantages of the task-based asynchronous pattern compared to the asynchronous pattern and the event-based asynchronous pattern available with earlier editions of .NET. You’ve also seen how easy it is to create asynchronous methods with the help of the Task class, and learned how to use the async and await keywords to wait for these methods without blocking threads. Finally, you looked at the error-handling aspect of asynchronous methods. For more information on parallel programming, and details about threads and tasks, see Chapter 21. The next chapter continues with core features of C# and .NET and gives detailed information on memory and resource management. c13.indd 346 30-01-2014 20:17:42 Memory Management and Pointers WHAT’s In THIs CHAPTER? ➤➤ Allocating space on the stack and heap at runtime ➤➤ Garbage collection ➤➤ Releasing unmanaged resources using destructors and the System .IDisposable interface ➤➤ The syntax for using pointers in C# ➤➤ Using pointers to implement high-performance stack-based arrays WRoX.CoM CodE doWnloAds FoR THIs CHAPTER The wrox.com code downloads for this chapter are found at www.wrox.com/go/procsharp on the Download Code tab. The code for this chapter is divided into the following major examples: ➤➤ PointerPlayground ➤➤ PointerPlayground2 ➤➤ QuickArray MEMoRy MAnAgEMEnT This chapter presents various aspects of memory management and memory access. Although the runtime removes much of the responsibility for memory management from the programmer, it is useful to understand how memory management works, and important to know how to work with unmanaged resources effi ciently. A good understanding of memory management and knowledge of the pointer capabilities provided by C# will better enable you to integrate C# code with legacy code and perform effi cient memory manip- ulation in performance-critical systems. 14 c14.indd 347 30-01-2014 20:18:30 348  ❘  CHAPTER 14  Memory Management and Pointers Memory Management Under the Hood One of the advantages of C# programming is that the programmer does not need to worry about detailed memory management; the garbage collector deals with the problem of memory cleanup on your behalf. As a result, you get something that approximates the efficiency of languages such as C++ without the complex- ity of having to handle memory management yourself as you do in C++. However, although you do not have to manage memory manually, it still pays to understand what is going on behind the scenes. Understanding how your program manages memory under the covers will help you increase the speed and performance of your applications. This section looks at what happens in the computer’s memory when you allocate variables. NOTE  The precise details of many of the topics of this section are not presented here. This section serves as an abbreviated guide to the general processes rather than as a statement of exact implementation. Value Data Types Windows uses a system known as virtual addressing, in which the mapping from the memory address seen by your program to the actual location in hardware memory is entirely managed by Windows. As a result, each process on a 32-bit processor sees 4GB of available memory, regardless of how much hardware mem- ory you actually have in your computer (on 64-bit processors this number is greater). This memory contains everything that is part of the program, including the executable code, any DLLs loaded by the code, and the contents of all variables used when the program runs. This 4GB of memory is known as the virtual address space or virtual memory. For convenience, this chapter uses the shorthand memory. Each memory location in the available 4GB is numbered starting from zero. To access a value stored at a particular location in memory, you need to supply the number that represents that memory location. In any compiled high-level language, including C#, Visual Basic, C++, and Java, the compiler converts human-read- able variable names into memory addresses that the processor understands. Somewhere inside a processor’s virtual memory is an area known as the stack. The stack stores value data types that are not members of objects. In addition, when you call a method, the stack is used to hold a copy of any parameters passed to the method. To understand how the stack works, you need to understand the importance of variable scope in C#. If variable a goes into scope before variable b, then b will always go out of scope first. Consider the following code: { int a; // do something { int b; // do something else } } First, a is declared. Then, inside the inner code block, b is declared. Then the inner code block terminates and b goes out of scope, then a goes out of scope. Therefore, the lifetime of b is entirely contained within the lifetime of a. The idea that you always de-allocate variables in the reverse order of how you allocate them is crucial to the way the stack works. Note that b is in a different block from code (defined by a different nesting of curly braces). For this reason, it is contained within a different scope. This is termed as block scope or structure scope. You do not know exactly where in the address space the stack is — you don’t need to know for C# develop- ment. A stack pointer (a variable maintained by the operating system) identifies the next free location on the c14.indd 348 30-01-2014 20:18:30 Memory Management Under the Hood  ❘  349 stack. When your program first starts running, the stack pointer will point to just past the end of the block of memory that is reserved for the stack. The stack fills downward, from high memory addresses to low addresses. As data is put on the stack, the stack pointer is adjusted accordingly, so it always points to just past the next free location. This is illustrated in Figure 14-1, which shows a stack pointer with a value of 800000 (0xC3500 in hex); the next free location is the address 799999. The following code tells the compiler that you need space in memory to store an integer and a double, and these memory locations are referred to as nRacingCars and engineSize. The line that declares each vari- able indicates the point at which you start requiring access to this variable. The closing curly brace of the block in which the variables are declared identifies the point at which both variables go out of scope: { int nRacingCars = 10; double engineSize = 3000.0; // do calculations; } Assuming that you use the stack shown in Figure 14-1, when the variable nRacingCars comes into scope and is assigned the value 10, the value 10 is placed in locations 799996 through 799999, the 4 bytes just below the location pointed to by the stack pointer (4 bytes because that’s how much memory is needed to store an int). To accommodate this, 4 is subtracted from the value of the stack pointer, so it now points to the location 799996, just after the new first free location (799995). The next line of code declares the variable engineSize (a double) and initializes it to the value 3000.0. A double occupies eight bytes, so the value 3000.0 is placed in locations 799988 through 799995 on the stack, and the stack pointer is decremented by eight, so that it again points to the location just after the next free location on the stack. When engineSize goes out of scope, the runtime knows that it is no longer needed. Because of the way variable lifetimes are always nested, you can guarantee that whatever happened while engineSize was in scope, the stack pointer is now pointing to the location where engineSize is stored. To remove engine- Size from the stack, the stack pointer is incremented by eight and it now points to the location immediately after the end of engineSize. At this point in the code, you are at the closing curly brace, so nRacingCars also goes out of scope. The stack pointer is incremented by 4. When another variable comes into scope after engineSize and nRacingCars have been removed from the stack, it overwrites the memory descending from location 799999, where nRacingCars was stored. If the compiler hits a line such as int i, j, then the order of variables coming into scope looks indetermi- nate. Both variables are declared at the same time and go out of scope at the same time. In this situation, it does not matter in what order the two variables are removed from memory. The compiler internally always ensures that the one that was put in memory first is removed last, thus preserving the rule that prohibits crossover of variable lifetimes. Reference Data Types Although the stack provides very high performance, it is not flexible enough to be used for all variables. The requirement that the lifetime of a variable must be nested is too restrictive for many purposes. Often, you need to use a method to allocate memory for storing data and keeping that data available long after that method has exited. This possibility exists whenever storage space is requested with the new operator — as is the case for all reference types. That is where the managed heap comes in. If you have done any C++ coding that required low-level memory management, you are familiar with the heap. The managed heap is not quite the same as the heap C++ uses, however; the managed heap works under the control of the garbage collector and provides significant benefits compared to traditional heaps. FIGURE 14-1 Location 800000 USED 799999 FREE Stack Pointer 799998 799997 c14.indd 349 30-01-2014 20:18:32 350  ❘  CHAPTER 14  Memory Management and Pointers The managed heap (or heap for short) is just another area of memory from the processor’s available memory. The following code demonstrates how the heap works and how memory is allocated for reference data types: void DoWork() { Customer arabel; arabel = new Customer(); Customer otherCustomer2 = new EnhancedCustomer(); } This code assumes the existence of two classes, Customer and EnhancedCustomer. The EnhancedCustomer class extends the Customer class. First, you declare a Customer reference called arabel. The space for this is allocated on the stack, but remember that this is only a reference, not an actual Customer object. The arabel reference occupies 4 bytes, enough space to hold the address at which a Customer object will be stored. (You need 4 bytes to represent a memory address as an integer value between 0 and 4GB.) The next line, arabel = new Customer(); does several things. First, it allocates memory on the heap to store a Customer object (a real object, not just an address). Then it sets the value of the variable arabel to the address of the memory it has allocated to the new Customer object. (It also calls the appropriate Customer constructor to initialize the fields in the class instance, but we won’t worry about that here.) The Customer instance is not placed on the stack — it is placed on the heap. In this example, you don’t know precisely how many bytes a Customer object occupies, but assume for the sake of argument that it is 32. These 32 bytes contain the instance fields of Customer as well as some information that .NET uses to identify and manage its class instances. To find a storage location on the heap for the new Customer object, the .NET runtime looks through the heap and grabs the first adjacent, unused block of 32 bytes. Again for the sake of argument, assume that this happens to be at address 200000, and that the arabel reference occupied locations 799996 through 799999 on the stack. This means that before instantiating the arabel object, the memory content will look similar to Figure 14-2. FIGURE 14-2 STACK HEAP FREE 200000 199999 USED USED 799996 – 799999 arabel FREE Stack Pointer After allocating the new Customer object, the content of memory will look like Figure 14-3. Note that unlike the stack, memory in the heap is allocated upward, so the free space can be found above the used space. c14.indd 350 30-01-2014 20:18:33 Memory Management Under the Hood  ❘  351 The next line of code both declares a Customer reference and instantiates a Customer object. In this instance, space on the stack for the otherCustomer2 reference is allocated and space for the mrJones object is allocated on the heap in a single line of code: Customer otherCustomer2 = new EnhancedCustomer(); This line allocates 4 bytes on the stack to hold the otherCustomer2 reference, stored at locations 799992 through 799995. The otherCustomer2 object is allocated space on the heap starting at location 200032. It is clear from the example that the process of setting up a reference variable is more complex than that for setting up a value variable, and there is performance overhead. In fact, the process is somewhat oversimpli- fied here, because the .NET runtime needs to maintain information about the state of the heap, and this information needs to be updated whenever new data is added to the heap. Despite this overhead, you now have a mechanism for allocating variables that is not constrained by the limitations of the stack. By assign- ing the value of one reference variable to another of the same type, you have two variables that reference the same object in memory. When a reference variable goes out of scope, it is removed from the stack as described in the previous section, but the data for a referenced object is still sitting on the heap. The data remains on the heap until either the program terminates or the garbage collector removes it, which happens only when it is no longer referenced by any variables. That is the power of reference data types, and you will see this feature used extensively in C# code. It means that you have a high degree of control over the lifetime of your data, because it is guaranteed to exist in the heap as long as you are maintaining some reference to it. Garbage Collection The previous discussion and diagrams show the managed heap working very much like the stack, to the extent that successive objects are placed next to each other in memory. This means that you can determine where to place the next object by using a heap pointer that indicates the next free memory location, which is adjusted as you add more objects to the heap. However, things are complicated by the fact that the lives of the heap-based objects are not coupled with the scope of the individual stack-based variables that reference them. When the garbage collector runs, it removes all those objects from the heap that are no longer referenced. Immediately after doing this, the heap will have objects scattered on it, mixed up with memory that has just been freed (see Figure 14-4). If the managed heap stayed like this, allocating space for new objects would be an awkward process, with the runtime having to search through the heap for a block of memory big enough to store each new object. FIGURE 14-3 STACK HEAP FREE 200032 200000 – 2000031 arabel instance 1999999 USED USED 799996 – 799999 arabel FREE Stack Pointer c14.indd 351 30-01-2014 20:18:34 352  ❘  CHAPTER 14  Memory Management and Pointers However, the garbage collector does not leave the heap in this state. As soon as the gar- bage collector has freed up all the objects it can, it compacts the heap by moving all the remaining objects to form one continuous block of memory. This means that the heap can continue working just like the stack, as far as locating where to store new objects. Of course, when the objects are moved about, all the references to those objects need to be updated with the correct new addresses, but the garbage collector handles that too. This action of compacting by the garbage collector is where the managed heap works very differently from old, unmanaged heaps. With the managed heap, it is just a ques- tion of reading the value of the heap pointer, rather than iterating through a linked list of addresses to find somewhere to put the new data. For this reason, instantiating an object under .NET is should be much faster. Interestingly, accessing objects tends to be faster too, because the objects are compacted toward the same area of memory on the heap, resulting in less page swapping. Microsoft believes that these performance gains more than compensate for the performance penalty you get whenever the garbage col- lector needs to do some work to compact the heap and change all those references to objects it has moved. NOTE  Generally, the garbage collector runs when the .NET runtime determines that garbage collection is required. You can force the garbage collector to run at a certain point in your code by calling System.GC.Collect. The System.GC class is a .NET class that represents the garbage collector, and the Collect method initiates a garbage collection. The GC class is intended for rare situations in which you know that it’s a good time to call the garbage collector; for example, if you have just de-referenced a large number of objects in your code. However, the logic of the garbage collector does not guarantee that all unreferenced objects will be removed from the heap in a single garbage collection pass. When objects are created, they are placed within the managed heap. The first section of the heap is called the generation 0 section, or gen 0. As your new objects are created, they are moved into this section of the heap. Therefore, this is where the youngest objects reside. Your objects remain there until the first collection of objects occurs through the garbage collection process. The objects that remain alive after this cleansing are compacted and then moved to the next section or gen- erational part of the heap — the generation 1, or gen 1, section. At this point, the generation 0 section is empty, and all new objects are again placed in this section. Older objects that survived the GC (garbage collection) process are found further down in the generation 1 sec- tion. This movement of aged items actually occurs one more time. The next collection process that occurs is then repeated. This means that the items that survived the GC process from the generation 1 section are moved to the generation 2 section, and the gen 0 items go to gen 1, again leaving gen 0 open for new objects. NOTE  Interestingly, a garbage collection will occur when you allocate an item that exceeds the capacity of the generation 0 section or when a GC.Collect is called. This process greatly improves the performance of your application. Typically, your youngest objects are the ones that can be collected, and a large number of younger-related objects might be reclaimed as well. If these objects reside next to each other in the heap, then the garbage collection process will be faster. In addition, because related objects are residing next to each other, program execution will be faster all around. Another performance-related aspect of garbage collection in .NET is how the framework deals with larger objects that are added to the heap. Under the covers of .NET, larger objects have their own managed heap, FIGURE 14-4 In use In use In use Free Free c14.indd 352 30-01-2014 20:18:36 Freeing Unmanaged Resources  ❘  353 referred to as the Large Object Heap. When objects greater than 85,000 bytes are utilized, they go to this special heap rather than the main heap. Your .NET application doesn’t know the difference, as this is all managed for you. Because compressing large items in the heap is expensive, it isn’t done for the objects resid- ing in the Large Object Heap. In an effort to improve GC even more, collections on the generation 2 section and from the Large Object Heap are now done on a background thread. This means that application threads are only blocked for gen- eration 0 and generation 1 collections, which reduces the overall pause time, especially for large-scale server apps. This feature is on by default for both servers and workstations. To turn it off, set the element in the configuration file to false. Another optimization to help in application performance is GC balancing. This is specific to server GC. Typically a server will have a pool of threads doing roughly the same thing. The memory allocation will be similar across all the threads. For servers there is one GC heap per logical server. So when one of the heaps runs out of memory and triggers a GC, all of the other heaps most likely will benefit from the GC as well. If a thread happens to use a lot more memory than other threads and it causes a GC, the other threads may not be close to requiring the GC so it’s not efficient. The GC will balance the heaps  —  both the Small Object Heap and also the Large Object Heap. By doing this balancing process, you can reduce unnecessary collection. To take advantage of hardware with lots of memory, the GC has added the GCSettings.LatencyMode property. Setting the property to one of the values in the GCLatencyMode enumeration will give a little con- trol to how the GC performs collections. Table 14-1 shows the possible values that can be used. Table 14-1:  Settings for GCLatencyMode Member Description Batch Disables the concurrency settings and sets the GC for maximum throughput. This will override the configura- tion setting. Interactive The default behavior. LowLatency Conservative GC. Full collections only occur when there is memory pressure on the system. Should only be used for short periods of time to perform specific operations. SustainedLowLatency Do full blocking collections only when there is system memory pressure. The amount of time that the LowLatency settings are used should be kept to a minimum. The amount of memory being allocated should be as small as possible. An out-of-memory error could occur if care is not taken. To take advantage of new high memory 64-bit machines, the configuration setting has been added. This will allow an object greater than 2 GB in size to be created. This will have no effect on 32-bit machines on which the 2GB limit is still in place. Freeing Unmanaged Resources The presence of the garbage collector means that you usually do not need to worry about objects you no longer need; you simply allow all references to those objects to go out of scope and let the garbage collector free memory as required. However, the garbage collector does not know how to free unmanaged resources (such as file handles, network connections, and database connections). When managed classes encapsulate direct or indirect references to unmanaged resources, you need to make special provisions to ensure that the unmanaged resources are released when an instance of the class is garbage collected. c14.indd 353 30-01-2014 20:18:36 354  ❘  CHAPTER 14  Memory Management and Pointers When defining a class, you can use two mechanisms to automate the freeing of unmanaged resources. These mechanisms are often implemented together because each provides a slightly different approach: ➤➤ Declare a destructor (or finalizer) as a member of your class. ➤➤ Implement the System.IDisposable interface in your class. The following sections discuss each of these mechanisms in turn, and then look at how to implement them together for best results. Destructors You have seen that constructors enable you to specify actions that must take place whenever an instance of a class is created. Conversely, destructors are called before an object is destroyed by the garbage collec- tor. Given this behavior, a destructor would initially seem like a great place to put code to free unmanaged resources and perform a general cleanup. Unfortunately, things are not so straightforward. NOTE  Although we talk about destructors in C#, in the underlying .NET architecture these are known as finalizers. When you define a destructor in C#, what is emitted into the assembly by the compiler is actually a Finalize method. It doesn’t affect any of your source code, but you need to be aware of it when examining the content of an assembly. The syntax for a destructor will be familiar to C++ developers. It looks like a method, with the same name as the containing class, but prefixed with a tilde (~). It has no return type, and takes no parameters or access modifiers. Here is an example: class MyClass { ~MyClass() { // destructor implementation } } When the C# compiler compiles a destructor, it implicitly translates the destructor code to the equivalent of a Finalize method, which ensures that the Finalize method of the parent class is executed. The following example shows the C# code equivalent to the Intermediate Language (IL) that the compiler would generate for the ~MyClass destructor: protected override void Finalize() { try { // destructor implementation } finally { base.Finalize(); } } As shown, the code implemented in the ~MyClass destructor is wrapped in a try block contained in the Finalize method. A call to the parent’s Finalize method is ensured by placing the call in a finally block. You can read about try and finally blocks in Chapter 16, “Errors and Exceptions.” Experienced C++ developers make extensive use of destructors, sometimes not only to clean up resources but also to provide debugging information or perform other tasks. C# destructors are used far less than their C++ equivalents. The problem with C# destructors as compared to their C++ counterparts is that they c14.indd 354 30-01-2014 20:18:36 Freeing Unmanaged Resources  ❘  355 are nondeterministic. When a C++ object is destroyed, its destructor runs immediately. However, because of the way the garbage collector works when using C#, there is no way to know when an object’s destructor will actually execute. Hence, you cannot place any code in the destructor that relies on being run at a cer- tain time, and you should not rely on the destructor being called for different class instances in any particu- lar order. When your object is holding scarce and critical resources that need to be freed as soon as possible, you do not want to wait for garbage collection. Another problem with C# destructors is that the implementation of a destructor delays the final removal of an object from memory. Objects that do not have a destructor are removed from memory in one pass of the garbage collector, but objects that have destructors require two passes to be destroyed: The first pass calls the destructor without removing the object, and the second pass actually deletes the object. In addition, the runtime uses a single thread to execute the Finalize methods of all objects. If you use destructors frequently, and use them to execute lengthy cleanup tasks, the impact on performance can be noticeable. The IDisposable Interface In C#, the recommended alternative to using a destructor is using the System.IDisposable interface. The IDisposable interface defines a pattern (with language-level support) that provides a deterministic mecha- nism for freeing unmanaged resources and avoids the garbage collector–related problems inherent with destructors. The IDisposable interface declares a single method named Dispose, which takes no param- eters and returns void. Here is an implementation for MyClass: class MyClass: IDisposable { public void Dispose() { // implementation } } The implementation of Dispose should explicitly free all unmanaged resources used directly by an object and call Dispose on any encapsulated objects that also implement the IDisposable interface. In this way, the Dispose method provides precise control over when unmanaged resources are freed. Suppose that you have a class named ResourceGobbler, which relies on the use of some external resource and implements IDisposable. If you want to instantiate an instance of this class, use it, and then dispose of it, you could do so like this: ResourceGobbler theInstance = new ResourceGobbler(); // do your processing theInstance.Dispose(); Unfortunately, this code fails to free the resources consumed by theInstance if an exception occurs during processing, so you should write the code as follows using a try block (as covered in detail in Chapter 16): ResourceGobbler theInstance = null; try { theInstance = new ResourceGobbler(); // do your processing } finally { if (theInstance != null) { theInstance.Dispose(); } } c14.indd 355 30-01-2014 20:18:36 356  ❘  CHAPTER 14  Memory Management and Pointers This version ensures that Dispose is always called on theInstance and that any resources consumed by it are always freed, even if an exception occurs during processing. However, if you always had to repeat such a construct, it would result in confusing code. C# offers a syntax that you can use to guarantee that Dispose is automatically called against an object that implements IDisposable when its reference goes out of scope. The syntax to do this involves the using keyword — though now in a very different context, which has nothing to do with namespaces. The following code generates IL code equivalent to the try block just shown: using (ResourceGobbler theInstance = new ResourceGobbler()) { // do your processing } The using statement, followed in brackets by a reference variable declaration and instantiation, causes that variable to be scoped to the accompanying statement block. In addition, when that variable goes out of scope, its Dispose method will be called automatically, even if an exception occurs. However, if you are already using try blocks to catch other exceptions, it is cleaner and avoids additional code indentation if you avoid the using statement and simply call Dispose in the finally clause of the existing try block. NOTE  For some classes, the notion of a Close method is more logical than Dispose, such as when dealing with files or database connections. In these cases, it is common to implement the IDisposable interface and then implement a separate Close method that simply calls Dispose. This approach provides clarity in the use of your classes and supports the using statement provided by C#. Implementing IDisposable and a Destructor The previous sections discussed two alternatives for freeing unmanaged resources used by the classes you create: ➤➤ The execution of a destructor is enforced by the runtime but is nondeterministic and places an unac- ceptable overhead on the runtime because of the way garbage collection works. ➤➤ The IDisposable interface provides a mechanism that enables users of a class to control when resources are freed but requires discipline to ensure that Dispose is called. In general, the best approach is to implement both mechanisms to gain the benefits of both while overcom- ing their limitations. You implement IDisposable on the assumption that most programmers will call Dispose correctly, but implement a destructor as a safety mechanism in case Dispose is not called. Here is an example of a dual implementation: using System; public class ResourceHolder: IDisposable { private bool isDisposed = false; public void Dispose() { Dispose(true); GC.SuppressFinalize(this); } protected virtual void Dispose(bool disposing) { if (!isDisposed) c14.indd 356 30-01-2014 20:18:36 Freeing Unmanaged Resources  ❘  357 { if (disposing) { // Cleanup managed objects by calling their // Dispose() methods. } // Cleanup unmanaged objects } isDisposed = true; } ~ResourceHolder() { Dispose (false); } public void SomeMethod() { // Ensure object not already disposed before execution of any method if(isDisposed) { throw new ObjectDisposedException("ResourceHolder"); } // method implementation... } } You can see from this code that there is a second protected overload of Dispose that takes one bool parameter — and this is the method that does all the cleaning up. Dispose(bool) is called by both the destructor and by IDisposable.Dispose. The point of this approach is to ensure that all cleanup code is in one place. The parameter passed to Dispose(bool) indicates whether Dispose(bool) has been invoked by the destructor or by IDisposable.Dispose — Dispose(bool) should not be invoked from anywhere else in your code. The idea is this: ➤➤ If a consumer calls IDisposable.Dispose, that consumer is indicating that all managed and unman- aged resources associated with that object should be cleaned up. ➤➤ If a destructor has been invoked, all resources still need to be cleaned up. However, in this case, you know that the destructor must have been called by the garbage collector and you should not attempt to access other managed objects because you can no longer be certain of their state. In this situation, the best you can do is clean up the known unmanaged resources and hope that any referenced man- aged objects also have destructors that will perform their own cleaning up. The isDisposed member variable indicates whether the object has already been disposed of and ensures that you do not try to dispose of member variables more than once. It also allows you to test whether an object has been disposed of before executing any instance methods, as shown in SomeMethod. This simplis- tic approach is not thread-safe and depends on the caller ensuring that only one thread is calling the method concurrently. Requiring a consumer to enforce synchronization is a reasonable assumption and one that is used repeatedly throughout the .NET class libraries (in the Collection classes, for example). Threading and synchronization are discussed in Chapter 21, “Tasks, Threads, and Synchronization.” Finally, IDisposable.Dispose contains a call to the method System.GC.SuppressFinalize. GC is the class that represents the garbage collector, and the SuppressFinalize method tells the garbage collector that a class no longer needs to have its destructor called. Because your implementation of Dispose has already done all the cleanup required, there’s nothing left for the destructor to do. Calling SuppressFinalize means that the garbage collector will treat that object as if it doesn’t have a destructor at all. c14.indd 357 30-01-2014 20:18:36 358  ❘  CHAPTER 14  Memory Management and Pointers Unsafe Code As you have just seen, C# is very good at hiding much of the basic memory management from the developer, thanks to the garbage collector and the use of references. However, sometimes you will want direct access to memory. For example, you might want to access a function in an external (non-.NET) DLL that requires a pointer to be passed as a parameter (as many Windows API functions do), or possibly for performance rea- sons. This section examines the C# facilities that provide direct access to the content of memory. Accessing Memory Directly with Pointers Although we are introducing pointers as if they were a new topic, in reality pointers are not new at all. You have been using references freely in your code, and a reference is simply a type-safe pointer. You have already seen how variables that represent objects and arrays actually store the memory address of where the corresponding data (the referent) is stored. A pointer is simply a variable that stores the address of some- thing else in the same way as a reference. The difference is that C# does not allow you direct access to the address contained in a reference variable. With a reference, the variable is treated syntactically as if it stores the actual content of the referent. C# references are designed to make the language simpler to use and to prevent you from inadvertently doing something that corrupts the contents of memory. With a pointer, however, the actual memory address is available to you. This gives you a lot of power to perform new kinds of operations. For example, you can add 4 bytes to the address in order to examine or even modify whatever data happens to be stored 4 bytes further in memory. There are two main reasons for using pointers: ➤➤ Backward compatibility — Despite all the facilities provided by the .NET runtime, it is still possible to call native Windows API functions, and for some operations this may be the only way to accomplish your task. These API functions are generally written in C++ or # and often require pointers as param- eters. However, in many cases it is possible to write the DllImport declaration in a way that avoids use of pointers — for example, by using the System.IntPtr class. ➤➤ Performance — On those occasions when speed is of the utmost importance, pointers can provide a route to optimized performance. If you know what you are doing, you can ensure that data is accessed or manipulated in the most efficient way. However, be aware that more often than not, there are other areas of your code where you can likely make the necessary performance improvements without resorting to using pointers. Try using a code profiler to look for the bottlenecks in your code — one is included with Visual Studio. Low-level memory access has a price. The syntax for using pointers is more complex than that for reference types, and pointers are unquestionably more difficult to use correctly. You need good programming skills and an excellent ability to think carefully and logically about what your code is doing to use pointers suc- cessfully. Otherwise, it is very easy to introduce subtle, difficult-to-find bugs into your program when using pointers. For example, it is easy to overwrite other variables, cause stack overflows, access areas of memory that don’t store any variables, or even overwrite information about your code that is needed by the .NET runtime, thereby crashing your program. In addition, if you use pointers your code must be granted a high level of trust by the runtime’s code access security mechanism or it will not be allowed to execute. Under the default code access security policy, this is only possible if your code is running on the local machine. If your code must be run from a remote location, such as the Internet, users must grant your code additional permissions for it to work. Unless the users trust you and your code, they are unlikely to grant these permissions. Code access security is discussed in more detail in Chapter 22, “Security.” Despite these issues, pointers remain a very powerful and flexible tool in the writing of efficient code. c14.indd 358 30-01-2014 20:18:37 Unsafe Code  ❘  359 Warning  We strongly advise against using pointers unnecessarily because your code will not only be harder to write and debug, but it will also fail the memory type safety checks imposed by the CLR. Writing Unsafe Code with the unsafe Keyword As a result of the risks associated with pointers, C# allows the use of pointers only in blocks of code that you have specifically marked for this purpose. The keyword to do this is unsafe. You can mark an indi- vidual method as being unsafe like this: unsafe int GetSomeNumber() { // code that can use pointers } Any method can be marked as unsafe, regardless of what other modifiers have been applied to it (for example, static methods or virtual methods). In the case of methods, the unsafe modifier applies to the method’s parameters, allowing you to use pointers as parameters. You can also mark an entire class or struct as unsafe, which means that all its members are assumed unsafe: unsafe class MyClass { // any method in this class can now use pointers } Similarly, you can mark a member as unsafe: class MyClass { unsafe int* pX; // declaration of a pointer field in a class } Or you can mark a block of code within a method as unsafe: void MyMethod() { // code that doesn't use pointers unsafe { // unsafe code that uses pointers here } // more 'safe' code that doesn't use pointers } Note, however, that you cannot mark a local variable by itself as unsafe: int MyMethod() { unsafe int *pX; // WRONG } If you want to use an unsafe local variable, you need to declare and use it inside a method or block that is unsafe. There is one more step before you can use pointers. The C# compiler rejects unsafe code unless you tell it that your code includes unsafe blocks. The flag to do this is unsafe. Hence, to compile a file named MySource.cs that contains unsafe blocks (assuming no other compiler options), the command is, csc /unsafe MySource.cs or: csc -unsafe MySource.cs c14.indd 359 30-01-2014 20:18:37 360  ❘  CHAPTER 14  Memory Management and Pointers NOTE  If you are using Visual Studio 2005, 2008, 2010, 2012, or 2013 you will also find the option to compile unsafe code in the Build tab of the project properties window. Pointer Syntax After you have marked a block of code as unsafe, you can declare a pointer using the following syntax: int* pWidth, pHeight; double* pResult; byte*[] pFlags; This code declares four variables: pWidth and pHeight are pointers to integers, pResult is a pointer to a double, and pFlags is an array of pointers to bytes. It is common practice to use the prefix p in front of names of pointer variables to indicate that they are pointers. When used in a variable declaration, the sym- bol * indicates that you are declaring a pointer (that is, something that stores the address of a variable of the specified type). NOTE  C++ developers should be aware of the syntax difference between C++ and C#. The C# statement int* pX, pY; corresponds to the C++ statement int *pX, *pY;. In C#, the * symbol is associated with the type, rather than the variable name. When you have declared variables of pointer types, you can use them in the same way as normal variables, but first you need to learn two more operators: ➤➤ & means take the address of, and converts a value data type to a pointer — for example int to *int. This operator is known as the address operator. ➤➤ * means get the content of this address, and converts a pointer to a value data type — for example, *float to float. This operator is known as the indirection operator (or the de-reference operator). You can see from these definitions that & and * have opposite effects. NOTE  You might be wondering how it is possible to use the symbols & and * in this manner because these symbols also refer to the operators of bitwise AND (&) and multi- plication (*). Actually, it is always possible for both you and the compiler to know what is meant in each case because with the pointer meanings, these symbols always appear as unary operators — they act on only one variable and appear in front of that variable in your code. By contrast, bitwise AND and multiplication are binary operators — they require two operands. The following code shows examples of how to use these operators: int x = 10; int* pX, pY; pX = &x; pY = pX; *pY = 20; You start by declaring an integer, x, with the value 10 followed by two pointers to integers, pX and pY. You then set pX to point to x (that is, you set the content of pX to the address of x). Then you assign the value of pX to pY, so that pY also points to x. Finally, in the statement *pY = 20, you assign the value 20 as the con- tents of the location pointed to by pY — in effect changing x to 20 because pY happens to point to x. Note that there is no particular connection between the variables pY and x. It is just that at the present time, pY happens to point to the memory location at which x is held. c14.indd 360 30-01-2014 20:18:37 Unsafe Code  ❘  361 To get a better understanding of what is going on, consider that the integer x is stored at memory loca- tions 0x12F8C4 through 0x12F8C7 (1243332 to 1243335 in decimal) on the stack (there are four locations because an int occupies 4 bytes). Because the stack allocates memory downward, this means that the vari- ables pX will be stored at locations 0x12F8C0 to 0x12F8C3, and pY will end up at locations 0x12F8BC to 0x12F8BF. Note that pX and pY also occupy 4 bytes each. That is not because an int occupies 4 bytes, but because on a 32-bit processor you need 4 bytes to store an address. With these addresses, after executing the previous code, the stack will look like Figure 14-5. NOTE  Although this process is illustrated with integers, which are stored consecutively on the stack on a 32-bit processor, this does not happen for all data types. The reason is because 32-bit processors work best when retrieving data from memory in 4-byte chunks. Memory on such machines tends to be divided into 4-byte blocks, and each block is sometimes known under Windows as a DWORD because this was the name of a 32-bit unsigned int in pre-.NET days. It is most efficient to grab DWORDs from memory — storing data across DWORD boundaries normally results in a hardware performance hit. For this reason, the .NET runtime normally pads out data types so that the memory they occupy is a multiple of 4. For example, a short occupies 2 bytes, but if a short is placed on the stack, the stack pointer will still be decremented by 4, not 2, so the next variable to go on the stack will still start at a DWORD boundary. You can declare a pointer to any value type (that is, any of the predefined types uint, int, byte, and so on, or to a struct). However, it is not possible to declare a pointer to a class or an array; this is because doing so could cause problems for the garbage collector. To work properly, the garbage collector needs to know exactly what class instances have been created on the heap, and where they are; but if your code started manipulating classes using pointers, you could very easily corrupt the information on the heap concerning classes that the .NET runtime maintains for the garbage collector. In this context, any data type that the garbage collector can access is known as a managed type. Pointers can only be declared as unmanaged types because the garbage collector cannot deal with them. Casting Pointers to Integer Types Because a pointer really stores an integer that represents an address, you won’t be surprised to know that the address in any pointer can be converted to or from any integer type. Pointer-to-integer-type conversions must be explicit. Implicit conversions are not available for such conversions. For example, it is perfectly legitimate to write the following: int x = 10; int* pX, pY; pX = &x; pY = pX; *pY = 20; uint y = (uint)pX; int* pD = (int*)y; The address held in the pointer pX is cast to a uint and stored in the variable y. You have then cast y back to an int* and stored it in the new variable pD. Hence, now pD also points to the value of x. The primary reason for casting a pointer value to an integer type is to display it. The Console.Write and Console.WriteLine methods do not have any overloads that can take pointers, but they will accept and display pointer values that have been cast to integer types: Console.WriteLine("Address is " + pX); // wrong -- will give a // compilation error Console.WriteLine("Address is " + (uint)pX); // OK x=20 (=0x14) pX=0x12F8C4 pY=012F8C40x12F8BC-0x12F8BF 0x12F8C0-0x12F8C3 0x12F8C4-0x12F8C7 FIGURE 14-5 c14.indd 361 30-01-2014 20:18:39 362  ❘  CHAPTER 14  Memory Management and Pointers You can cast a pointer to any of the integer types. However, because an address occupies 4 bytes on 32-bit systems, casting a pointer to anything other than a uint, long, or ulong is almost certain to lead to over- flow errors. (An int causes problems because its range is from roughly –2 billion to 2 billion, whereas an address runs from zero to about 4 billion.) When C# is released for 64-bit processors, an address will occupy 8 bytes. Hence, on such systems, casting a pointer to anything other than ulong is likely to lead to overflow errors. It is also important to be aware that the checked keyword does not apply to conversions involving pointers. For such conversions, exceptions will not be raised when overflows occur, even in a checked context. The .NET runtime assumes that if you are using pointers, you know what you are doing and are not worried about possible overflows. Casting Between Pointer Types You can also explicitly convert between pointers pointing to different types. For example, the following is perfectly legal code: byte aByte = 8; byte* pByte= &aByte; double* pDouble = (double*)pByte; However, if you try something like this, be careful. In this example, if you look at the double value pointed to by pDouble, you will actually be looking up some memory that contains a byte (aByte), combined with some other memory, and treating it as if this area of memory contained a double, which will not give you a meaningful value. However, you might want to convert between types to implement the equivalent of a C union, or you might want to cast pointers from other types into pointers to sbyte to examine individual bytes of memory. void Pointers If you want to maintain a pointer but not specify to what type of data it points, you can declare it as a pointer to a void: int* pointerToInt; void* pointerToVoid; pointerToVoid = (void*)pointerToInt; The main use of this is if you need to call an API function that requires void* parameters. Within the C# language, there isn’t a great deal that you can do using void pointers. In particular, the compiler will flag an error if you attempt to de-reference a void pointer using the * operator. Pointer Arithmetic It is possible to add or subtract integers to and from pointers. However, the compiler is quite clever about how it arranges this. For example, suppose that you have a pointer to an int and you try to add 1 to its value. The compiler will assume that you actually mean you want to look at the memory location following the int, and hence it will increase the value by 4 bytes — the size of an int. If it is a pointer to a double, adding 1 will actually increase the value of the pointer by 8 bytes, the size of a double. Only if the pointer points to a byte or sbyte (1 byte each), will adding 1 to the value of the pointer actually change its value by 1. You can use the operators +, -, +=, -=, ++, and -- with pointers, with the variable on the right side of these operators being a long or ulong. NOTE  It is not permitted to carry out arithmetic operations on void pointers. c14.indd 362 30-01-2014 20:18:39 Unsafe Code  ❘  363 For example, assume the following definitions: uint u = 3; byte b = 8; double d = 10.0; uint* pUint= &u; // size of a uint is 4 byte* pByte = &b; // size of a byte is 1 double* pDouble = &d; // size of a double is 8 Next, assume the addresses to which these pointers point are as follows: ➤➤ pUint: 1243332 ➤➤ pByte: 1243328 ➤➤ pDouble: 1243320 Then execute this code: ++pUint; // adds (1*4) = 4 bytes to pUint pByte -= 3; // subtracts (3*1) = 3 bytes from pByte double* pDouble2 = pDouble + 4; // pDouble2 = pDouble + 32 bytes (4*8 bytes) The pointers now contain this: ➤➤ pUint: 1243336 ➤➤ pByte: 1243325 ➤➤ pDouble2: 1243352 NOTE  The general rule is that adding a number X to a pointer to type T with value P gives the result P + X*(sizeof(T)). If successive values of a given type are stored in successive memory locations, pointer addition works very well, allowing you to move pointers between memory locations. If you are dealing with types such as byte or char, though, with sizes not in multiples of 4, successive values will not, by default, be stored in successive memory locations. You can also subtract one pointer from another pointer, if both pointers point to the same data type. In this case, the result is a long whose value is given by the difference between the pointer values divided by the size of the type that they represent: double* pD1 = (double*)1243324; // note that it is perfectly valid to // initialize a pointer like this. double* pD2 = (double*)1243300; long L = pD1-pD2; // gives the result 3 (=24/sizeof(double)) The sizeof Operator This section has been referring to the size of various data types. If you need to use the size of a type in your code, you can use the sizeof operator, which takes the name of a data type as a parameter and returns the number of bytes occupied by that type, as shown in this example: int x = sizeof(double); This will set x to the value 8. The advantage of using sizeof is that you don’t have to hard-code data type sizes in your code, making your code more portable. For the predefined data types, sizeof returns the following values: sizeof(sbyte) = 1; sizeof(byte) = 1; sizeof(short) = 2; sizeof(ushort) = 2; sizeof(int) = 4; sizeof(uint) = 4; sizeof(long) = 8; sizeof(ulong) = 8; sizeof(char) = 2; sizeof(float) = 4; sizeof(double) = 8; sizeof(bool) = 1; c14.indd 363 30-01-2014 20:18:39 364  ❘  CHAPTER 14  Memory Management and Pointers You can also use sizeof for structs that you define yourself, although in that case, the result depends on what fields are in the struct. You cannot use sizeof for classes. Pointers to Structs: The Pointer Member Access Operator Pointers to structs work in exactly the same way as pointers to the predefined value types. There is, however, one condition — the struct must not contain any reference types. This is due to the restriction mentioned earlier that pointers cannot point to any reference types. To avoid this, the compiler will flag an error if you create a pointer to any struct that contains any reference types. Suppose that you had a struct defined like this: struct MyStruct { public long X; public float F; } You could define a pointer to it as follows: MyStruct* pStruct; Then you could initialize it like this: MyStruct Struct = new MyStruct(); pStruct = &Struct; It is also possible to access member values of a struct through the pointer: (*pStruct).X = 4; (*pStruct).F = 3.4f; However, this syntax is a bit complex. For this reason, C# defines another operator that enables you to access members of structs through pointers using a simpler syntax. It is known as the pointer member access operator, and the symbol is a dash followed by a greater-than sign, so it looks like an arrow: ->. NOTE  C++ developers will recognize the pointer member access operator because C++ uses the same symbol for the same purpose. Using the pointer member access operator, the previous code can be rewritten like this: pStruct->X = 4; pStruct->F = 3.4f; You can also directly set up pointers of the appropriate type to point to fields within a struct, long* pL = &(Struct.X); float* pF = &(Struct.F); or: long* pL = &(pStruct->X); float* pF = &(pStruct->F); Pointers to Class Members As indicated earlier, it is not possible to create pointers to classes. That is because the garbage collector does not maintain any information about pointers, only about references, so creating pointers to classes could cause garbage collection to not work properly. However, most classes do contain value type members, and you might want to create pointers to them. This is possible but requires a special syntax. For example, suppose that you rewrite the struct from the previous example as a class: c14.indd 364 30-01-2014 20:18:39 Unsafe Code  ❘  365 class MyClass { public long X; public float F; } Then you might want to create pointers to its fields, X and F, in the same way as you did earlier. Unfortunately, doing so will produce a compilation error: MyClass myObject = new MyClass(); long* pL = &(myObject.X); // wrong -- compilation error float* pF = &(myObject.F); // wrong -- compilation error Although X and F are unmanaged types, they are embedded in an object, which sits on the heap. During gar- bage collection, the garbage collector might move MyObject to a new location, which would leave pL and pF pointing to the wrong memory addresses. Because of this, the compiler will not let you assign addresses of members of managed types to pointers in this manner. The solution is to use the fixed keyword, which tells the garbage collector that there may be pointers ref- erencing members of certain objects, so those objects must not be moved. The syntax for using fixed looks like this if you just want to declare one pointer: MyClass myObject = new MyClass(); fixed (long* pObject = &(myObject.X)) { // do something } You define and initialize the pointer variable in the brackets following the keyword fixed. This pointer variable (pObject in the example) is scoped to the fixed block identified by the curly braces. As a result, the garbage collector knows not to move the myObject object while the code inside the fixed block is executing. If you want to declare more than one pointer, you can place multiple fixed statements before the same code block: MyClass myObject = new MyClass(); fixed (long* pX = &(myObject.X)) fixed (float* pF = &(myObject.F)) { // do something } You can nest entire fixed blocks if you want to fix several pointers for different periods: MyClass myObject = new MyClass(); fixed (long* pX = &(myObject.X)) { // do something with pX fixed (float* pF = &(myObject.F)) { // do something else with pF } } You can also initialize several variables within the same fixed block, if they are of the same type: MyClass myObject = new MyClass(); MyClass myObject2 = new MyClass(); fixed (long* pX = &(myObject.X), pX2 = &(myObject2.X)) { // etc. } In all these cases, it is immaterial whether the various pointers you are declaring point to fields in the same or different objects or to static fields not associated with any class instance. c14.indd 365 30-01-2014 20:18:39 366  ❘  CHAPTER 14  Memory Management and Pointers Pointer Example: PointerPlayground This section presents an example that uses pointers. The following code is an example named PointerPlayground. It does some simple pointer manipulation and displays the results, enabling you to see what is happening in memory and where variables are stored: using System; namespace PointerPlayground { class MainEntryPoint { static unsafe void Main() { int x=10; short y = -1; byte y2 = 4; double z = 1.5; int* pX = &x; short* pY = &y; double* pZ = &z; Console.WriteLine( "Address of x is 0x{0:X}, size is {1}, value is {2}", (uint)&x, sizeof(int), x); Console.WriteLine( "Address of y is 0x{0:X}, size is {1}, value is {2}", (uint)&y, sizeof(short), y); Console.WriteLine( "Address of y2 is 0x{0:X}, size is {1}, value is {2}", (uint)&y2, sizeof(byte), y2); Console.WriteLine( "Address of z is 0x{0:X}, size is {1}, value is {2}", (uint)&z, sizeof(double), z); Console.WriteLine( "Address of pX=&x is 0x{0:X}, size is {1}, value is 0x{2:X}", (uint)&pX, sizeof(int*), (uint)pX); Console.WriteLine( "Address of pY=&y is 0x{0:X}, size is {1}, value is 0x{2:X}", (uint)&pY, sizeof(short*), (uint)pY); Console.WriteLine( "Address of pZ=&z is 0x{0:X}, size is {1}, value is 0x{2:X}", (uint)&pZ, sizeof(double*), (uint)pZ); *pX = 20; Console.WriteLine("After setting *pX, x = {0}", x); Console.WriteLine("*pX = {0}", *pX); pZ = (double*)pX; Console.WriteLine("x treated as a double = {0}", *pZ); Console.ReadLine(); } } } This code declares four value variables: ➤➤ An int x ➤➤ A short y ➤➤ A byte y2 ➤➤ A double z c14.indd 366 30-01-2014 20:18:40 Unsafe Code  ❘  367 It also declares pointers to three of these values: pX, pY, and pZ. Next, you display the value of these variables as well as their size and address. Note that in taking the address of pX, pY, and pZ, you are effectively looking at a pointer to a pointer — an address of an address of a value. Also, in accordance with the usual practice when displaying addresses, you have used the {0:X} format specifier in the Console.WriteLine commands to ensure that memory addresses are displayed in hexadecimal format. Finally, you use the pointer pX to change the value of x to 20 and do some pointer casting to see what hap- pens if you try to treat the content of x as if it were a double. Compiling and running this code results in the following output. This screen output demonstrates the effects of attempting to compile both with and without the /unsafe flag: csc PointerPlayground.cs Microsoft (R) Visual C# Compiler version 4.0.30319.17379 for Microsoft(R) .NET Framework 4.5 Copyright (C) Microsoft Corporation. All rights reserved. PointerPlayground.cs(7,26): error CS0227: Unsafe code may only appear if compiling with /unsafe csc /unsafe PointerPlayground.cs Microsoft (R) Visual C# Compiler version 4.0.30319.17379 for Microsoft(R) .NET Framework 4.5 Copyright (C) Microsoft Corporation. All rights reserved. PointerPlayground Address of x is 0x12F4B0, size is 4, value is 10 Address of y is 0x12F4AC, size is 2, value is -1 Address of y2 is 0x12F4A8, size is 1, value is 4 Address of z is 0x12F4A0, size is 8, value is 1.5 Address of pX=&x is 0x12F49C, size is 4, value is 0x12F4B0 Address of pY=&y is 0x12F498, size is 4, value is 0x12F4AC Address of pZ=&z is 0x12F494, size is 4, value is 0x12F4A0 After setting *pX, x = 20 *pX = 20 x treated as a double = 2.86965129997082E-308 Checking through these results confirms the description of how the stack operates presented in the “Memory Management Under the Hood” section earlier in this chapter. It allocates successive variables moving downward in memory. Notice how it also confirms that blocks of memory on the stack are always allocated in multiples of 4 bytes. For example, y is a short (of size 2), and has the (decimal) address 1242284, indicating that the memory locations reserved for it are locations 1242284 through 1242287. If the .NET runtime had been strictly packing up variables next to each other, Y would have occupied just two locations, 1242284 and 1242285. The next example illustrates pointer arithmetic, as well as pointers to structs and class members. This exam- ple is named PointerPlayground2. To start, you define a struct named CurrencyStruct, which represents a currency value as dollars and cents. You also define an equivalent class named CurrencyClass: internal struct CurrencyStruct { public long Dollars; public byte Cents; public override string ToString() { return "$" + Dollars + "." + Cents; } } internal class CurrencyClass c14.indd 367 30-01-2014 20:18:40 368  ❘  CHAPTER 14  Memory Management and Pointers { public long Dollars; public byte Cents; public override string ToString() { return "$" + Dollars + "." + Cents; } } Now that you have your struct and class defined, you can apply some pointers to them. Following is the code for the new example. Because the code is fairly long, we will go through it in detail. You start by displaying the size of CurrencyStruct, creating a couple of CurrencyStruct instances and creating some CurrencyStruct pointers. You use the pAmount pointer to initialize the members of the amount1 CurrencyStruct and then display the addresses of your variables: public static unsafe void Main() { Console.WriteLine( "Size of CurrencyStruct struct is " + sizeof(CurrencyStruct)); CurrencyStruct amount1, amount2; CurrencyStruct* pAmount = &amount1; long* pDollars = &(pAmount->Dollars); byte* pCents = &(pAmount->Cents); Console.WriteLine("Address of amount1 is 0x{0:X}", (uint)&amount1); Console.WriteLine("Address of amount2 is 0x{0:X}", (uint)&amount2); Console.WriteLine("Address of pAmount is 0x{0:X}", (uint)&pAmount); Console.WriteLine("Address of pDollars is 0x{0:X}", (uint)&pDollars); Console.WriteLine("Address of pCents is 0x{0:X}", (uint)&pCents); pAmount->Dollars = 20; *pCents = 50; Console.WriteLine("amount1 contains " + amount1); Now you do some pointer manipulation that relies on your knowledge of how the stack works. Due to the order in which the variables were declared, you know that amount2 will be stored at an address immediately below amount1. The sizeof(CurrencyStruct) operator returns 16 (as demonstrated in the screen output coming up), so CurrencyStruct occupies a multiple of 4 bytes. Therefore, after you decrement your cur- rency pointer, it points to amount2: --pAmount; // this should get it to point to amount2 Console.WriteLine("amount2 has address 0x{0:X} and contains {1}", (uint)pAmount, *pAmount); Notice that when you call Console.WriteLine, you display the contents of amount2, but you haven’t yet initialized it. What is displayed will be random garbage — whatever happened to be stored at that location in memory before execution of the example. There is an important point here: normally, the C# compiler would prevent you from using an uninitialized variable, but when you start using pointers, it is very easy to circumvent many of the usual compilation checks. In this case, you have done so because the compiler has no way of knowing that you are actually displaying the contents of amount2. Only you know that, because your knowledge of the stack means that you can tell what the effect of decrementing pAmount will be. Once you start doing pointer arithmetic, you will find that you can access all sorts of variables and memory loca- tions that the compiler would usually stop you from accessing, hence the description of pointer arithmetic as unsafe. Next, you do some pointer arithmetic on your pCents pointer. pCents currently points to amount1.Cents, but the aim here is to get it to point to amount2.Cents, again using pointer operations instead of directly telling the compiler that’s what you want to do. To do this, you need to decrement the address pCents con- tains by sizeof(Currency): c14.indd 368 30-01-2014 20:18:40 Unsafe Code  ❘  369 // do some clever casting to get pCents to point to cents // inside amount2 CurrencyStruct* pTempCurrency = (CurrencyStruct*)pCents; pCents = (byte*) ( --pTempCurrency ); Console.WriteLine("Address of pCents is now 0x{0:X}", (uint)&pCents); Finally, you use the fixed keyword to create some pointers that point to the fields in a class instance and use these pointers to set the value of this instance. Notice that this is also the first time that you have been able to look at the address of an item stored on the heap, rather than the stack: Console.WriteLine("\nNow with classes"); // now try it out with classes CurrencyClass amount3 = new CurrencyClass(); fixed(long* pDollars2 = &(amount3.Dollars)) fixed(byte* pCents2 = &(amount3.Cents)) { Console.WriteLine( "amount3.Dollars has address 0x{0:X}", (uint)pDollars2); Console.WriteLine( "amount3.Cents has address 0x{0:X}", (uint) pCents2); *pDollars2 = -100; Console.WriteLine("amount3 contains " + amount3); } Compiling and running this code gives output similar to this: csc /unsafe PointerPlayground2.cs Microsoft (R) Visual C# 2010 Compiler version 4.0.21006.1 Copyright (C) Microsoft Corporation. All rights reserved. PointerPlayground2 Size of CurrencyStruct struct is 16 Address of amount1 is 0x12F4A4 Address of amount2 is 0x12F494 Address of pAmount is 0x12F490 Address of pDollars is 0x12F48C Address of pCents is 0x12F488 amount1 contains $20.50 amount2 has address 0x12F494 and contains $0.0 Address of pCents is now 0x12F488 Now with classes amount3.Dollars has address 0xA64414 amount3.Cents has address 0xA6441C amount3 contains $-100.0 Notice in this output the uninitialized value of amount2 that is displayed, and notice that the size of the CurrencyStruct struct is 16 — somewhat larger than you would expect given the size of its fields (a long and a byte should total 9 bytes). Using Pointers to Optimize Performance Until now, all the examples have been designed to demonstrate the various things that you can do with pointers. We have played around with memory in a way that is probably interesting only to people who like to know what’s happening under the hood, but that doesn’t really help you write better code. Now you’re going to apply your understanding of pointers and see an example of how judicious use of pointers has a significant performance benefit. Creating Stack-based Arrays This section explores one of the main areas in which pointers can be useful: creating high-performance, low- overhead arrays on the stack. As discussed in Chapter 2, “Core C#,” C# includes rich support for handling c14.indd 369 30-01-2014 20:18:40 370  ❘  CHAPTER 14  Memory Management and Pointers arrays. Although C# makes it very easy to use both 1-dimensional and rectangular or jagged multidimen- sional arrays, it suffers from the disadvantage that these arrays are actually objects; they are instances of System.Array. This means that the arrays are stored on the heap, with all the overhead that this involves. There may be occasions when you need to create a short-lived, high-performance array and don’t want the overhead of reference objects. You can do this by using pointers, although as you see in this section, this is easy only for 1-dimensional arrays. To create a high-performance array, you need to use a new keyword: stackalloc. The stackalloc command instructs the .NET runtime to allocate an amount of memory on the stack. When you call stackalloc, you need to supply it with two pieces of information: ➤➤ The type of data you want to store ➤➤ The number of these data items you need to store For example, to allocate enough memory to store 10 decimal data items, you can write the following: decimal* pDecimals = stackalloc decimal[10]; This command simply allocates the stack memory; it does not attempt to initialize the memory to any default value. This is fine for the purpose of this example because you are creating a high-performance array, and initializing values unnecessarily would hurt performance. Similarly, to store 20 double data items, you write this: double* pDoubles = stackalloc double[20]; Although this line of code specifies the number of variables to store as a constant, this can equally be a quantity evaluated at runtime. Therefore, you can write the previous example like this: int size; size = 20; // or some other value calculated at runtime double* pDoubles = stackalloc double[size]; You can see from these code snippets that the syntax of stackalloc is slightly unusual. It is followed imme- diately by the name of the data type you want to store (which must be a value type) and then by the number of items you need space for, in square brackets. The number of bytes allocated will be this number multi- plied by sizeof(data type). The use of square brackets in the preceding code sample suggests an array, which is not too surprising. If you have allocated space for 20 doubles, then what you have is an array of 20 doubles. The simplest type of array that you can have is a block of memory that stores one element after another (see Figure 14-6). This diagram also shows the pointer returned by stackalloc, which is always a pointer to the allocated data type that points to the top of the newly allocated memory block. To use the memory block, you simply de-reference the returned pointer. For example, to allocate space for 20 doubles and then set the first ele- ment (element 0 of the array) to the value 3.0, write this: double* pDoubles = stackalloc double[20]; *pDoubles = 3.0; To access the next element of the array, you use pointer arithmetic. As described earlier, if you add 1 to a pointer, its value will be increased by the size of whatever data type it points to. In this case, that’s just enough to take you to the next free memory location in the block that you have allocated. Therefore, you can set the second element of the array (e