Java并发编程实践


            JavaConcurrency InPractice BrianGöetz TimPeierls JoshuaBloch JosephBowbeer DavidHolmes DougLea                     AddisonͲWesleyProfessional ISBNͲ10:0Ͳ321Ͳ34960Ͳ1 ISBNͲ13:978Ͳ0Ͳ321Ͳ34960Ͳ6  ii JavaConcurrencyInPractice Index Index ii Preface xiii HowtoUsethisBook xiii CodeExamples xiv Acknowledgments xv Chapter 1 - Introduction 1 1.1. A (Very) Brief History of Concurrency 2 1.2. Benefits of Threads 3 1.2.1.ExploitingMultipleProcessors 3 1.2.2.SimplicityofModeling 3 1.2.3.SimplifiedHandlingofAsynchronousEvents 3 1.2.4.MoreResponsiveUserInterfaces 4 1.3. Risks of Threads 5 1.3.1.SafetyHazards 5 1.3.2.LivenessHazards 6 1.3.3.PerformanceHazards 6 1.4. Threads are Everywhere 8 Part I: Fundamentals 10 Chapter 2. Thread Safety 11 2.1.WhatisThreadSafety? 12 2.2.Atomicity 13 2.3.Locking 16 2.4.GuardingStatewithLocks 19 2.5.LivenessandPerformance 20 Chapter 3. Sharing Objects 23 3.1.Visibility 23 3.2.PublicationandEscape 26 3.3.ThreadConfinement 28 3.4.Immutability 31 3.5.SafePublication 33 Chapter 4. Composing Objects 37 4.1.DesigningaThreadͲsafeClass 37 4.2.InstanceConfinement 39 4.3.DelegatingThreadSafety 41 4.4.AddingFunctionalitytoExistingThreadͲsafeClasses 47 4.5.DocumentingSynchronizationPolicies 49 Chapter 5. Building Blocks 51 5.1.SynchronizedCollections 51 5.2.ConcurrentCollections 54 5.3.BlockingQueuesandtheProducerͲconsumerPattern 56 5.4.BlockingandInterruptibleMethods 59 5.5.Synchronizers 60 5.6.BuildinganEfficient,ScalableResultCache 64 SummaryofPartI 69  iii> void sort(List list) { // Never returns the wrong answer! System.exit(0); } Somereadersmayquestiontheroleofthe"bad"examplesinthisbook;afterall,abookshouldshowhowtodothings right,notwrong.Thebadexampleshavetwopurposes.Theyillustratecommonpitfalls,butmoreimportantlythey demonstratehowtoanalyzeaprogramforthreadsafetyͲandthebestwaytodothatistoseethewaysinwhich threadsafetyiscompromised.  xv2BPreface Listing2.LessthanOptimalWaytoSortaList. public > void sort(List list) { for (int i=0; i<1000000; i++) doNothing(); Collections.sort(list); } Acknowledgments Thisbookgrewoutofthedevelopmentprocessforthejava.util.concurrentpackagethatwascreatedbytheJava CommunityProcessJSR166forinclusioninJava5.0.ManyotherscontributedtoJSR166;inparticularwethankMartin Buchholz for doing all the work related to getting the code into the JDK, and all the readers of the concurrency- interestmailinglistwhoofferedtheirsuggestionsandfeedbackonthedraftAPIs. Thisbookhasbeentremendouslyimprovedbythesuggestionsandassistanceofasmallarmyofreviewers,advisors, cheerleaders,andarmchaircritics.WewouldliketothankDionAlmaer,TracyBialik,CindyBloch,MartinBuchholz,Paul Christmann,CliffClick,StuartHalloway,DavidHovemeyer,JasonHunter,MichaelHunter,JeremyHylton,HeinzKabutz, Robert Kuhar, Ramnivas Laddad, Jared Levy, Nicole Lewis, Victor Luchangco, Jeremy Manson, Paul Martin, Berna Massingill,MichaelMaurer,TedNeward,KirkPepperdine,BillPugh,SamPullara,RussRufer,BillScherer,JeffreySiegal, Bruce Tate, Gil Tene, Paul Tyma, and members of the Silicon Valley Patterns Group who, through many interesting technicalconversations,offeredguidanceandmadesuggestionsthathelpedmakethisbookbetter. WeareespeciallygratefultoCliffBiffle,BarryHayes,DawidKurzyniec,AngelikaLanger,DoronRajwan,andBillVenners, whoreviewedtheentiremanuscriptinexcruciatingdetail,foundbugsinthecodeexamples,andsuggestednumerous improvements. WethankKatrinaAveryforagreatcopyͲeditingjobandRosemarySimpsonforproducingtheindexunderunreasonable timepressure.WethankAmiDewarfordoingtheillustrations. ThankstothewholeteamatAddisonͲWesleywhohelpedmakethisbookareality.AnnSellersgottheprojectlaunched andGregDoenchshepherdedittoasmoothcompletion;ElizabethRyanguideditthroughtheproductionprocess. Wewouldalsoliketothankthethousandsofsoftwareengineerswhocontributedindirectlybycreatingthesoftware usedtocreatethisbook,includingTEX,LATEX,AdobeAcrobat,pic,grap,AdobeIllustrator,Perl,ApacheAnt,IntelliJ IDEA,GNUemacs,Subversion,TortoiseSVN,andofcourse,theJavaplatformandclasslibraries.   13BChapter1Ͳ Introduction Ͳ 10B1.1.A(Very)BriefHistoryofConcurrency Chapter1ǦIntroduction Writingcorrectprogramsishard;writingcorrectconcurrentprogramsisharder.Therearesimplymorethingsthatcan gowronginaconcurrentprogramthaninasequentialone.So,whydowebotherwithconcurrency?Threadsarean inescapable feature of the Java language, and they can simplify the development of complex systems by turning complicated asynchronous code into simpler straightͲline code. In addition, threads are the easiest way to tap the computingpowerofmultiprocessorsystems.And,asprocessorcountsincrease,exploitingconcurrencyeffectivelywill onlybecomemoreimportant.  2 JavaConcurrencyInPractice 1.1.A(Very)BriefHistoryofConcurrency Intheancientpast,computersdidn'thaveoperatingsystems;theyexecutedasingleprogramfrombeginningtoend, andthatprogramhaddirectaccesstoalltheresourcesofthemachine.Notonlywasitdifficulttowriteprogramsthat ran on the bare metal, but runningonly a single program at a time was an inefficient use of expensive and scarce computerresources. Operatingsystemsevolvedtoallowmorethanoneprogramtorunatonce,runningindividualprogramsinprocesses: isolated,independentlyexecutingprogramstowhichtheoperatingsystemallocatesresourcessuchasmemory,file handles,andsecuritycredentials.Iftheyneededto,processescouldcommunicatewithoneanotherthroughavariety ofcoarseͲgrainedcommunicationmechanisms:sockets,signalhandlers,sharedmemory,semaphores,andfiles. Several motivating factors led to the development of operating systems that allowed multiple programs to execute simultaneously: Resource utilization. Programs sometimes have to wait for external operations such as input or output, and while waitingcandonousefulwork.Itismoreefficienttousethatwaittimetoletanotherprogramrun. Fairness.Multipleusersandprogramsmayhaveequalclaimsonthemachine'sresources.Itispreferabletoletthem sharethecomputerviafinerͲgrainedtimeslicingthantoletoneprogramruntocompletionandthenstartanother. Convenience.Itisofteneasierormoredesirabletowriteseveralprogramsthateachperformasingletaskandhave themcoordinatewitheachotherasnecessarythantowriteasingleprogramthatperformsallthetasks. Inearlytimesharingsystems,eachprocesswasavirtualvonNeumanncomputer;ithadamemoryspacestoringboth instructions and data, executing instructions sequentially according to the semantics of the machine language, and interacting with the outside world via the operating system through a set of I/O primitives. For each instruction executedtherewasaclearlydefined"nextinstruction",andcontrolflowedthroughtheprogramaccordingtotherules oftheinstructionset.Nearlyallwidelyusedprogramminglanguagestodayfollowthissequentialprogrammingmodel, wherethelanguagespecificationclearlydefines"whatcomesnext"afteragivenactionisexecuted. Thesequentialprogrammingmodelisintuitiveandnatural,asitmodelsthewayhumanswork:doonethingatatime, in sequence mostly. Get out of bed, put on your bathrobe, go downstairs and start the tea. As in programming languages, each of these realͲworld actions is an abstraction for a sequence of finerͲgrained actionsͲopen the cupboard,selectaflavoroftea,measuresometeaintothepot,seeifthere'senoughwaterintheteakettle,ifnotput somemorewaterin,setitonthestove,turnthestoveon,waitforthewatertoboil,andsoon.ThislaststepͲwaiting forthewatertoboilͲalsoinvolvesadegreeofasynchrony.Whilethewaterisheating,youhaveachoiceofwhattodoͲ just wait, or do other tasks in that time such as starting the toast (another asynchronous task) or fetching the newspaper, while remaining aware that your attention will soon be needed by the teakettle. The manufacturers of teakettlesandtoastersknowtheirproductsareoftenusedinanasynchronousmanner,sotheyraiseanaudiblesignal whentheycompletetheirtask.Findingtherightbalanceofsequentialityandasynchronyisoftenacharacteristicof efficientpeopleͲandthesameistrueofprograms. Thesameconcerns(resourceutilization,fairness,andconvenience)thatmotivatedthedevelopmentofprocessesalso motivated the development of threads. Threads allow multiple streams of program control flow to coexist within a process.TheyshareprocessͲwideresourcessuchasmemoryandfilehandles,buteachthreadhasitsownprogram counter,stack,andlocalvariables.Threadsalsoprovideanaturaldecompositionforexploitinghardwareparallelismon multiprocessorsystems;multiplethreadswithinthesameprogramcanbescheduledsimultaneouslyonmultipleCPUs. Threadsaresometimescalledlightweightprocesses,andmostmodernoperatingsystemstreatthreads,notprocesses, as the basic units of scheduling. In the absence of explicit coordination, threads execute simultaneously and asynchronouslywithrespecttooneanother.Sincethreadssharethememoryaddressspaceoftheirowningprocess,all threadswithinaprocesshaveaccesstothesamevariablesandallocateobjectsfromthesameheap,whichallowsfinerͲ grained data sharing than interͲprocess mechanisms. But without explicit synchronization to coordinate access to shareddata,athreadmaymodifyvariablesthatanotherthreadisinthemiddleofusing,withunpredictableresults.  33BChapter1Ͳ IntroductionͲ11B1.2.BenefitsofThreads 1.2.BenefitsofThreads Whenusedproperly,threadscanreducedevelopmentandmaintenancecostsandimprovetheperformanceofcomplex applications.Threadsmakeiteasiertomodelhowhumansworkandinteract,byturningasynchronousworkflowsinto mostlysequentialones.TheycanalsoturnotherwiseconvolutedcodeintostraightͲlinecodethatiseasiertowrite, read,andmaintain. ThreadsareusefulinGUIapplicationsforimprovingtheresponsivenessoftheuserinterface,andinserverapplications for improving resource utilization and throughput. They also simplify the implementation of the JVMͲthe garbage collector usually runs in one or more dedicated threads. Most nontrivial Java applications rely to some degree on threadsfortheirorganization. 1.2.1.ExploitingMultipleProcessors Multiprocessor systems used to be expensive and rare, found only in large data centers and scientific computing facilities.Todaytheyarecheapandplentiful;evenlowͲendserverandmidrangedesktopsystemsoftenhavemultiple processors.Thistrendwillonlyaccelerate;asitgetshardertoscaleupclockrates,processormanufacturerswillinstead putmoreprocessorcoresonasinglechip.Allthemajorchipmanufacturershavebegunthistransition,andweare alreadyseeingmachineswithdramaticallyhigherprocessorcounts. Sincethebasicunitofschedulingisthethread,aprogramwithonlyonethreadcanrunonatmostoneprocessorata time.OnatwoͲprocessorsystem,asingleͲthreadedprogramisgivingupaccesstohalftheavailableCPUresources;ona 100Ͳprocessor system, it is giving up access to 99%. On the other hand, programs with multiple active threads can execute simultaneously on multiple processors. When properly designed, multithreaded programs can improve throughputbyutilizingavailableprocessorresourcesmoreeffectively. Using multiple threads can also help achieve better throughput on singleͲprocessor systems. If a program is singleͲ threaded,theprocessorremainsidlewhileitwaitsforasynchronousI/Ooperationtocomplete.Inamultithreaded program,anotherthreadcanstillrunwhilethefirstthreadiswaitingfortheI/Otocomplete,allowingtheapplicationto stillmakeprogressduringtheblockingI/O.(Thisislikereadingthenewspaperwhilewaitingforthewatertoboil,rather thanwaitingforthewatertoboilbeforestartingtoread.) 1.2.2.SimplicityofModeling Itisofteneasiertomanageyourtimewhenyouhaveonlyonetypeoftasktoperform(fixthesetwelvebugs)than whenyouhaveseveral(fixthebugs,interviewreplacementcandidatesforthesystemadministrator,completeyour team'sperformanceevaluations,andcreatetheslidesforyourpresentationnextweek).Whenyouhaveonlyonetype oftasktodo,youcanstartatthetopofthepileandkeepworkinguntilthepileisexhausted(oryouare);youdon't havetospendanymentalenergyfiguringoutwhattoworkonnext.Ontheotherhand,managingmultiplepriorities anddeadlinesandswitchingfromtasktotaskusuallycarriessomeoverhead. Thesameistrueforsoftware:aprogramthatprocessesonetypeoftasksequentiallyissimplertowrite,lesserrorͲ prone,andeasiertotestthanonemanagingmultipledifferenttypesoftasksatonce.Assigningathreadtoeachtypeof taskortoeachelementinasimulationaffordstheillusionofsequentialityandinsulatesdomainlogicfromthedetailsof scheduling,interleavedoperations,asynchronousI/O,andresourcewaits.Acomplicated,asynchronousworkflowcan bedecomposedintoanumberofsimpler,synchronousworkflowseachrunninginaseparatethread,interactingonly witheachotheratspecificsynchronizationpoints. Thisbenefitisoftenexploitedbyframeworkssuch asservletsorRMI (Remote MethodInvocation).Theframework handlesthedetailsofrequestmanagement,threadcreation,andloadbalancing,dispatchingportionsoftherequest handlingtotheappropriateapplicationcomponentattheappropriatepointintheworkͲflow.Servletwritersdonot needtoworryabouthowmanyotherrequestsarebeingprocessedatthesametimeorwhetherthesocketinputand output streams block; when a servlet's service method is called in response to a web request, it can process the requestsynchronouslyasifitwereasingleͲthreadedprogram.Thiscansimplifycomponentdevelopmentandreduce thelearningcurveforusingsuchframeworks. 1.2.3.SimplifiedHandlingofAsynchronousEvents Aserverapplicationthatacceptssocketconnectionsfrommultipleremoteclientsmaybeeasiertodevelopwheneach connectionisallocateditsownthreadandallowedtousesynchronousI/O.  4 JavaConcurrencyInPractice Ifanapplicationgoestoreadfromasocketwhennodataisavailable,readblocksuntilsomedataisavailable.Ina singleͲthreadedapplication,thismeansthatnotonlydoesprocessingthecorrespondingrequeststall,butprocessingof all requests stalls while the single thread is blocked. To avoid this problem, singleͲthreaded server applications are forcedtousenonͲblockingI/O,whichisfarmorecomplicatedanderrorͲpronethansynchronousI/O.However,ifeach requesthasitsownthread,thenblockingdoesnotaffecttheprocessingofotherrequests. Historically,operatingsystemsplacedrelativelylowlimitsonthenumberofthreadsthataprocesscouldcreate,asfew asseveralhundred(orevenless).Asaresult,operatingsystemsdevelopedefficientfacilitiesformultiplexedI/O,such as the Unix select and poll system calls, and to access these facilities, the Java class libraries acquired a set of packages (java.nio) for nonͲblocking I/O. However, operating system support for larger numbers of threads has improved significantly, making the threadͲperͲclient model practical even for large numbers of clients on some platforms.[1] [1]TheNPTLthreadspackage,nowpartofmostLinuxdistributions,wasdesignedtosupporthundredsofthousandsofthreads.NonͲblockingI/O hasitsownbenefits,butbetterOSsupportforthreadsmeansthattherearefewersituationsforwhichitisessential. 1.2.4.MoreResponsiveUserInterfaces GUIapplicationsusedtobesingleͲthreaded,whichmeantthatyouhadtoeitherfrequentlypollthroughoutthecodefor inputevents(whichismessyandintrusive)orexecuteallapplicationcodeindirectlythrougha"maineventloop".If codecalledfromthemaineventlooptakestoolongtoexecute,theuserinterfaceappearsto"freeze"untilthatcode finishes,becausesubsequentuserinterfaceeventscannotbeprocesseduntilcontrolisreturnedtothemaineventloop. Modern GUI frameworks,such as the AWT and Swing toolkits, replace the main event loop with anevent dispatch thread(EDT).Whenauserinterfaceeventsuchasabuttonpressoccurs,applicationͲdefinedeventhandlersarecalled intheeventthread.MostGUIframeworksaresingleͲthreadedsubsystems,sothemaineventloopiseffectivelystill present,butitrunsinitsownthreadunderthecontroloftheGUItoolkitratherthantheapplication. IfonlyshortͲlivedtasksexecuteintheeventthread,theinterfaceremainsresponsivesincetheeventthreadisalways abletoprocessuseractionsreasonablyquickly.However,processingalongͲrunningtaskintheeventthread,suchas spellͲcheckingalargedocumentorfetchingaresourceoverthenetwork,impairsresponsiveness.Iftheuserperforms anactionwhilethistaskisrunning,thereisalongdelaybeforetheeventthreadcanprocessorevenacknowledgeit.To addinsulttoinjury,notonlydoestheUIbecomeunresponsive,butitisimpossibletocanceltheoffendingtaskevenif theUIprovidesacancelbuttonbecausetheeventthreadisbusyandcannothandlethecancelbuttonͲpresseventuntil the lengthy task completes! If, however, the longͲrunning task is instead executed in a separate thread, the event threadremainsfreetoprocessUIevents,makingtheUImoreresponsive.  53BChapter1Ͳ IntroductionͲ12B1.3.RisksofThreads 1.3.RisksofThreads Java's builtͲin support for threads is a doubleͲedged sword. While it simplifies the development of concurrent applications byproviding languageandlibrarysupportandaformalcrossͲplatformmemorymodel (it isthisformal crossͲplatform memory model that makes possible the development of writeͲonce, runͲanywhere concurrent applicationsinJava),italsoraisesthebarfordevelopersbecausemoreprogramswillusethreads.Whenthreadswere more esoteric, concurrency was an "advanced" topic; now, mainstream developers must be aware of threadͲsafety issues. 1.3.1.SafetyHazards Thread safety can be unexpectedly subtle because, in the absence of sufficient synchronization, the ordering of operations in multiple threads is unpredictable and sometimes surprising. UnsafeSequence in Listing 1.1, which is supposedtogenerateasequenceofuniqueintegervalues,offersasimpleillustrationofhowtheinterleavingofactions in multiple threads can lead to undesirable results. It behaves correctly in a singleͲthreaded environment, but in a multithreadedenvironmentdoesnot. Listing1.1.NonǦthreadǦsafeSequenceGenerator. @NotThreadSafe public class UnsafeSequence { private int value; /** Returns a unique value. */ public int getNext() { return value++; } } TheproblemwithUnsafeSequenceisthatwithsomeunluckytiming,twothreadscouldcallgetNextandreceivethe samevalue.Figure1.1showshowthiscanhappen.Theincrementnotation,nextValue++,mayappeartobeasingle operation,butisinfactthreeseparateoperations:readthevalue,addonetoit,andwriteoutthenewvalue.Since operationsinmultiplethreadsmaybearbitrarilyinterleavedbytheruntime,itispossiblefortwothreadstoreadthe valueatthesametime,bothseethesamevalue,andthenbothaddonetoit.Theresultisthatthesamesequence numberisreturnedfrommultiplecallsindifferentthreads. Figure1.1.UnluckyExecutionofUnsafeSequence.Nextvalue.   DiagramslikeFigure1.1depictpossibleinterleavingsofoperationsindifferentthreads.Inthesediagrams,timeruns fromlefttoright,andeachlinerepresentstheactivitiesofadifferentthread.Theseinterleavingdiagramsusuallydepict theworstcase[2]andareintendedtoshowthedangerofincorrectlyassumingthingswillhappeninaparticularorder. [2]Actually,aswe'llseeinChapter3,theworstcasecanbeevenworsethanthesediagramsusuallyshowbecauseofthepossibilityofreordering. UnsafeSequence uses a nonstandard annotation: @NotThreadSafe. This is one of several custom annotations used throughoutthisbooktodocumentconcurrencypropertiesofclassesandclassmembers.(OtherclassͲlevelannotations usedinthiswayare@ThreadSafeand@Immutable;seeAppendixAfordetails.)Annotationsdocumentingthreadsafety are useful to multiple audiences. If a class is annotated with @ThreadSafe, users can use it with confidence in a multithreaded environment, maintainers are put on notice that it makes thread safety guarantees that must be preserved,andsoftwareanalysistoolscanidentifypossiblecodingerrors.  6 JavaConcurrencyInPractice UnsafeSequenceillustratesacommonconcurrencyhazardcalledaracecondition.WhetherornotnextValuereturnsa unique value when called from multiple threads, as required by its specification, depends on how the runtime interleavestheoperationsͲwhichisnotadesirablestateofaffairs. Becausethreadssharethesamememoryaddressspaceandrunconcurrently,theycanaccessormodifyvariablesthat otherthreadsmightbeusing.Thisisatremendousconvenience,becauseitmakesdatasharingmucheasierthanwould otherinterͲthreadcommunicationsmechanisms.Butitisalsoasignificantrisk:threadscanbeconfusedbyhavingdata changeunexpectedly.AllowingmultiplethreadstoaccessandmodifythesamevariablesintroducesanelementofnonͲ sequentialityintoanotherwisesequentialprogrammingmodel,whichcanbeconfusinganddifficulttoreasonabout. Foramultithreadedprogram'sbehaviortobepredictable,accesstosharedvariablesmustbeproperlycoordinatedso thatthreadsdonotinterferewithoneanother.Fortunately,Javaprovidessynchronizationmechanismstocoordinate suchaccess. UnsafeSequencecanbefixedbymakinggetNextasynchronizedmethod,asshowninSequenceinListing1.2,[3]thus preventingtheunfortunateinteractioninFigure1.1.(ExactlywhythisworksisthesubjectofChapters2and3.) [3]@GuardedByisdescribedinSection2.4;itdocumentsthesynchronizationpolicyforSequence. Listing1.2.ThreadǦsafeSequenceGenerator. @ThreadSafe public class Sequence { @GuardedBy("this") private int nextValue; public synchronized int getNext() { return nextValue++; } } Intheabsenceofsynchronization,thecompiler,hardware,andruntimeareallowedtotakesubstantiallibertieswith the timing and ordering of actions, such as caching variables in registers or processorͲlocal caches where they are temporarily (or even permanently) invisible to other threads. These tricks are in aid ofbetter performance and are generally desirable, but they place a burden on the developer to clearly identify where data is being shared across threads so that these optimizations do not undermine safety. (Chapter 16 gives the gory details on exactly what orderingguaranteestheJVMmakesandhowsynchronizationaffectsthoseguarantees,butifyoufollowtherulesin Chapters2and3,youcansafelyavoidtheselowͲleveldetails.) 1.3.2.LivenessHazards Itiscriticallyimportanttopayattentiontothreadsafetyissueswhendevelopingconcurrentcode:safetycannotbe compromised.TheimportanceofsafetyisnotuniquetomultithreadedprogramsͲsingleͲthreadedprogramsalsomust takecaretopreservesafetyandcorrectnessͲbuttheuseofthreadsintroducesadditionalsafetyhazardsnotpresentin singleͲthreadedprograms.Similarly,theuseofthreadsintroducesadditionalformsoflivenessfailurethatdonotoccur insingleͲthreadedprograms. While safety means "nothing bad ever happens", liveness concerns the complementary goal that "something good eventuallyhappens".Alivenessfailureoccurswhenanactivitygetsintoastatesuchthatitispermanentlyunableto makeforwardprogress.Oneformoflivenessfailurethatcanoccurinsequentialprogramsisaninadvertentinfinite loop,wherethecodethatfollowstheloopnevergetsexecuted.Theuseofthreadsintroducesadditionallivenessrisks. Forexample,ifthreadAiswaitingforaresourcethatthreadBholdsexclusively,andBneverreleasesit,Awillwait forever.Chapter10describesvariousformsoflivenessfailuresandhowtoavoidthem,includingdeadlock(Section 10.1),starvation(Section10.3.1),andlivelock(Section10.3.3).Likemostconcurrencybugs,bugsthatcauseliveness failurescanbeelusivebecausetheydependontherelativetimingofeventsindifferentthreads,andthereforedonot alwaysmanifestthemselvesindevelopmentortesting. 1.3.3.PerformanceHazards Relatedtolivenessisperformance.Whilelivenessmeansthatsomethinggoodeventuallyhappens,eventuallymaynot be good enoughͲwe often want good things to happen quickly. Performance issues subsume a broad range of problems,includingpoorservicetime,responsiveness,throughput,resourceconsumption,orscalability.Justaswith safetyandliveness,multithreadedprogramsaresubjecttoalltheperformancehazardsofsingleͲthreadedprograms, andtoothersaswellthatareintroducedbytheuseofthreads.  73BChapter1Ͳ IntroductionͲ12B1.3.RisksofThreads Inwelldesignedconcurrentapplicationstheuseofthreadsisanetperformancegain,butthreadsneverthelesscarry somedegreeofruntimeoverhead.ContextswitchesͲwhentheschedulersuspendstheactivethreadtemporarilyso anotherthreadcanrunͲaremorefrequentinapplicationswithmanythreads,andhavesignificantcosts:savingand restoringexecutioncontext,lossoflocality,andCPUtimespentschedulingthreadsinsteadofrunningthem.When threads share data, they must use synchronization mechanisms that can inhibit compiler optimizations, flush or invalidatememorycaches,andcreatesynchronizationtrafficonthesharedmemorybus.Allthesefactorsintroduce additionalperformancecosts;Chapter11coverstechniquesforanalyzingandreducingthesecosts.  8 JavaConcurrencyInPractice 1.4.ThreadsareEverywhere Evenifyourprogramneverexplicitlycreatesathread,frameworksmaycreatethreadsonyourbehalf,andcodecalled fromthesethreadsmustbethreadͲsafe.Thiscanplaceasignificantdesignandimplementationburdenondevelopers, sincedevelopingthreadͲsafeclassesrequiresmorecareandanalysisthandevelopingnonͲthreadͲsafeclasses. Every Java application uses threads. When the JVM starts, it creates threads for JVM housekeeping tasks (garbage collection,finalization)andamainthreadforrunningthemainmethod.TheAWT(AbstractWindowToolkit)andSwing user interface frameworks create threads for managing user interface events. Timer creates threads for executing deferred tasks. Component frameworks, such as servlets and RMI create pools of threads and invoke component methodsinthesethreads. IfyouusethesefacilitiesͲasmanydevelopersdoͲyouhavetobefamiliarwithconcurrencyandthreadsafety,because theseframeworkscreatethreadsandcallyourcomponentsfromthem.Itwouldbenicetobelievethatconcurrencyis an"optional"or"advanced"languagefeature,buttherealityisthatnearlyallJavaapplicationsaremultithreadedand theseframeworksdonotinsulateyoufromtheneedtoproperlycoordinateaccesstoapplicationstate. Whenconcurrencyisintroducedintoanapplicationbyaframework,itisusuallyimpossibletorestricttheconcurrencyͲ awarenesstotheframeworkcode,becauseframeworksbytheirnaturemakecallbackstoapplicationcomponentsthat inturnaccessapplicationstate.Similarly,theneedforthreadsafetydoesnotendwiththecomponentscalledbythe frameworkͲitextendstoallcodepathsthataccesstheprogramstateaccessedbythosecomponents.Thus,theneed forthreadsafetyiscontagious. Frameworks introduce concurrency into applications by calling application components from framework threads. Componentsinvariablyaccessapplicationstate,thusrequiringthatallcodepathsaccessingthatstatebethreadͲsafe. Thefacilitiesdescribedbelowallcauseapplicationcodetobe calledfromthreadsnot managed by theapplication. While the needforthreadsafetymaystartwiththesefacilities,itrarelyendsthere;instead,itripplesthroughthe application. Timer.Timerisaconveniencemechanismforschedulingtaskstorunatalatertime,eitheronceorperiodically.The introductionofaTimercancomplicateanotherwisesequentialprogram,becauseTimerTasksareexecutedinathread managedbythe Timer,nottheapplication.Ifa TimerTaskaccessesdatathatisalsoaccessedbyotherapplication threads,thennotonlymusttheTimerTaskdosoinathreadͲsafemanner,butsomustanyotherclassesthataccess thatdata.OftentheeasiestwaytoachievethisistoensurethatobjectsaccessedbytheTimerTaskarethemselves threadͲsafe,thusencapsulatingthethreadsafetywithinthesharedobjects. ServletsandJavaServerPages(JSPs).Theservletsframeworkisdesignedtohandlealltheinfrastructureofdeployinga web application and dispatching requests from remote HTTP clients. A request arriving at the server is dispatched, perhapsthroughachainoffilters,totheappropriateservletorJSP.Eachservletrepresentsacomponentofapplication logic,andinhighͲvolumewebsites,multipleclientsmayrequiretheservicesofthesameservletatonce.Theservlets specification requiresthataservletbepreparedtobecalledsimultaneouslyfrommultiplethreads.Inotherwords, servletsneedtobethreadͲsafe. Even if you could guarantee that a servlet was only called from one thread at a time, you would still have to pay attentiontothreadsafetywhenbuildingawebapplication.Servletsoftenaccessstateinformationsharedwithother servlets, such as applicationͲscoped objects (those stored in the ServletContext) or sessionͲscoped objects (those stored in the perͲclient HttpSession). When a servlet accesses objects shared across servlets or requests, it must coordinate access to these objects properly, since multiple requests could be accessing them simultaneously from separate threads. Servlets and JSPs, as well as servlet filters and objects stored in scoped containers like ServletContextandHttpSession,simplyhavetobethreadͲsafe. RemoteMethodInvocation.RMIletsyouinvokemethodsonobjectsrunninginanotherJVM.Whenyoucallaremote methodwithRMI,themethodargumentsarepackaged(marshaled)intoabytestreamandshippedoverthenetworkto theremoteJVM,wheretheyareunpacked(unmarshaled)andpassedtotheremotemethod. WhentheRMIcodecallsyourremoteobject,inwhatthreaddoesthatcallhappen?Youdon'tknow,butit'sdefinitely notinathreadyoucreatedͲyourobjectgetscalledinathreadmanagedbyRMI.HowmanythreadsdoesRMIcreate? CouldthesameremotemethodonthesameremoteobjectbecalledsimultaneouslyinmultipleRMIthreads?[4]  93BChapter1Ͳ Introduction Ͳ 13B1.4.ThreadsareEverywhere [4]Answer:yes,butit'snotallthatclearfromtheJavadocͲyouhavetoreadtheRMIspec. A remote object must guard against two thread safety hazards: properly coordinating access to state that may be sharedwithotherobjects,andproperlycoordinatingaccesstothestateoftheremoteobjectitself(sincethesame objectmaybecalledinmultiplethreadssimultaneously).Likeservlets,RMIobjectsshouldbepreparedformultiple simultaneouscallsandmustprovidetheirownthreadsafety. SwingandAWT.GUIapplicationsareinherentlyasynchronous.Usersmayselectamenuitemorpressabuttonatany time,andtheyexpectthattheapplicationwillrespondpromptlyevenifitisinthemiddleofdoingsomethingelse. SwingandAWTaddressthisproblembycreatingaseparatethreadforhandlinguserͲinitiatedeventsandupdatingthe graphicalviewpresentedtotheuser. Swing components, such as JTable, are not threadͲsafe. Instead, Swing programs achieve their thread safety by confiningallaccesstoGUIcomponentstotheeventthread.IfanapplicationwantstomanipulatetheGUIfromoutside theeventthread,itmustcausethecodethatwillmanipulatetheGUItorunintheeventthreadinstead. WhentheuserperformsaUIaction,aneventhandleriscalledintheeventthreadtoperformwhateveroperationthe userrequested.Ifthehandlerneedstoaccessapplicationstatethatisalsoaccessedfromotherthreads(suchasa documentbeingedited),thentheeventhandler,alongwithanyothercodethataccessesthatstate,mustdosoina threadͲsafemanner.   10 JavaConcurrencyInPractice PartI:Fundamentals  Chapter2.ThreadSafety Chapter3.SharingObjects Chapter4.ComposingObjects Chapter5.BuildingBlocks  11 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety Chapter2.ThreadSafety Perhapssurprisingly,concurrentprogrammingisn'tsomuchaboutthreadsorlocks,anymorethancivilengineeringis aboutrivetsandIͲbeams.Ofcourse,buildingbridgesthatdon'tfalldownrequiresthecorrectuseofalotofrivetsandIͲ beams,justasbuildingconcurrentprogramsrequirethecorrectuseofthreadsandlocks.Butthesearejustmechanisms meanstoanend.WritingthreadͲsafecodeis,atitscore,aboutmanagingaccesstostate,andinparticulartoshared, mutablestate.  Informally,anobject'sstateisitsdata,storedinstatevariablessuchasinstanceorstaticfields.Anobject'sstatemay includefieldsfromother,dependentobjects;aHashMap'sstateispartiallystoredintheHashMapobjectitself,butalsoin manyMap.Entryobjects.Anobject'sstateencompassesanydatathatcanaffectitsexternallyvisiblebehavior. Byshared,wemeanthatavariablecouldbeaccessedbymultiplethreads;bymutable,wemeanthatitsvaluecould changeduringitslifetime.Wemaytalkaboutthreadsafetyasifitwereaboutcode,butwhatwearereallytryingtodo isprotectdatafromuncontrolledconcurrentaccess. WhetheranobjectneedstobethreadͲsafedependsonwhetheritwillbeaccessedfrommultiplethreads.Thisisa property of how the object is used in a program, not what it does. Making an object threadͲsafe requires using synchronizationtocoordinateaccesstoitsmutablestate;failingtodosocouldresultindatacorruptionandother undesirableconsequences. Whenever more than one thread accesses a given state variable, and one of them might write to it, they all must coordinate their access to it using synchronization. The primary mechanism for synchronization in Java is the synchronized keyword, which provides exclusive locking, but the term "synchronization" also includes the use of volatilevariables,explicitlocks,andatomicvariables. Youshouldavoidthetemptationtothinkthatthereare"special"situationsinwhichthisruledoesnotapply.Aprogram thatomitsneededsynchronizationmightappeartowork,passingitstestsandperformingwellforyears,butitisstill brokenandmayfailatanymoment. If multiple threads access the same mutable state variable without appropriate synchronization, your program is broken.Therearethreewaystofixit: x Don'tsharethestatevariableacrossthreads; x Makethestatevariableimmutable;or x Usesynchronizationwheneveraccessingthestatevariable. If you haven't considered concurrent access in your class design, some of these approaches can require significant designmodifications,sofixingtheproblemmightnotbeastrivialasthisadvicemakesitsound.Itisfareasiertodesign aclasstobethreadͲsafethantoretrofititforthreadsafetylater. Inalargeprogram,identifyingwhethermultiplethreadsmightaccessagivenvariablecanbecomplicated.Fortunately, thesameobjectͲorientedtechniquesthathelpyouwritewellͲorganized,maintainableclassesͲsuchasencapsulation anddatahidingͲcanalsohelpyoucreatethreadͲsafeclasses.Thelesscodethathasaccesstoaparticularvariable,the easieritistoensurethatallofitusesthepropersynchronization,andtheeasieritistoreasonabouttheconditions under which a given variable might be accessed. The Java language doesn't force you to encapsulate stateͲit is perfectly allowable to store state in public fields (even public static fields) or publish a reference to an otherwise internalobjectͲbutthebetterencapsulatedyourprogramstate,theeasieritistomakeyourprogramthreadͲsafeand tohelpmaintainerskeepitthatway. When designing threadͲsafe classes, good objectͲoriented techniquesͲencapsulation, immutability, and clear specificationofinvariantsͲareyourbestfriends. TherewillbetimeswhengoodobjectͲorienteddesigntechniquesareatoddswithrealͲworldrequirements;itmaybe necessary in these cases to compromise the rules of good design for the sake of performance or for the sake of backward compatibility with legacy code. Sometimes abstraction and encapsulation are at odds with performanceͲ  12 JavaConcurrencyInPractice althoughnotnearlyasoftenasmanydevelopersbelieveͲbutitisalwaysagoodpracticefirsttomakeyourcoderight, andthenmakeitfast.Eventhen,pursueoptimizationonlyifyourperformancemeasurementsandrequirementstell youthatyoumust,andifthosesamemeasurementstellyouthatyouroptimizationsactuallymadeadifferenceunder realisticconditions.[1] [1]Inconcurrentcode,thispracticeshouldbeadheredtoevenmorethanusual.Becauseconcurrencybugsaresodifficulttoreproduceand debug,thebenefitofasmallperformancegainonsomeinfrequentlyusedcodepathmaywellbedwarfedbytheriskthattheprogramwillfailin thefield. Ifyoudecidethatyousimplymustbreakencapsulation,allisnotlost.ItisstillpossibletomakeyourprogramthreadͲ safe, it is just a lot harder. Moreover, the thread safety of your program will be more fragile, increasing not only developmentcostandriskbutmaintenancecostandriskaswell.Chapter4characterizestheconditionsunderwhichit issafetorelaxencapsulationofstatevariables. We'veusedtheterms"threadͲsafeclass"and"threadͲsafeprogram"nearlyinterchangeablythusfar.IsathreadͲsafe programonethatisconstructedentirelyofthreadͲsafeclasses?NotnecessarilyͲaprogramthatconsistsentirelyof threadͲsafeclassesmaynotbethreadͲsafe,andathreadͲsafeprogrammaycontainclassesthatarenotthreadͲsafe. TheissuessurroundingthecompositionofthreadͲsafeclassesarealsotakenupinChapter4.Inanycase,theconceptof athreadͲsafeclassmakessenseonlyiftheclassencapsulatesitsownstate.Threadsafetymaybeatermthatisapplied tocode,butitisaboutstate,anditcanonlybeappliedtotheentirebodyofcodethatencapsulatesitsstate,whichmay beanobjectoranentireprogram. 2.1.WhatisThreadSafety? Definingthreadsafetyissurprisinglytricky.Themoreformalattemptsaresocomplicated astoofferlittlepractical guidanceorintuitiveunderstanding,andtherestareinformaldescriptionsthatcanseemdownrightcircular.Aquick Googlesearchturnsupnumerous"definitions"likethese: ...canbecalledfrommultipleprogramthreadswithoutunwantedinteractionsbetweenthethreads. ...maybecalledbymorethanonethreadatatimewithoutrequiringanyotheractiononthecaller'spart. Given definitionslikethese,it'snowonderwefindthreadsafetyconfusing! Theysound suspiciouslylike"a classis threadͲsafeifitcanbeusedsafelyfrommultiplethreads."Youcan'treallyarguewithsuchastatement,butitdoesn't offermuchpracticalhelpeither.HowdowetellathreadͲsafeclassfromanunsafeone?Whatdoweevenmeanby "safe"? Attheheartofanyreasonabledefinitionofthreadsafetyistheconceptofcorrectness.Ifourdefinitionofthreadsafety isfuzzy,itisbecausewelackacleardefinitionofcorrectness. Correctness means that a class conforms to its specification. A good specification defines invariants constraining an object's state and postͲconditions describing the effects of its operations. Since we often don't write adequate specificationsforourclasses,howcanwepossiblyknowtheyarecorrect?Wecan't,butthatdoesn'tstopusfromusing themanywayoncewe'veconvincedourselvesthat"thecodeworks".This"codeconfidence"isaboutascloseasmany ofusgettocorrectness,solet'sjustassumethatsingleͲthreadedcorrectnessissomethingthat"weknowitwhenwe seeit".Havingoptimisticallydefined"correctness"assomethingthatcanberecognized,wecannowdefinethread safetyinasomewhatlesscircularway:aclassisthreadͲsafewhenitcontinuestobehavecorrectlywhenaccessedfrom multiplethreads. A class is threadͲsafe if it behaves correctly when accessed from multiple threads, regardless of the scheduling or interleavingoftheexecutionofthosethreadsbytheruntimeenvironment,andwithnoadditionalsynchronizationor othercoordinationonthepartofthecallingcode. Since any singleͲthreaded program is also a valid multithreaded program, it cannot be threadͲsafe if it is not even correctinasingleͲthreadedenvironment.[2]Ifanobjectiscorrectlyimplemented,nosequenceofoperationsͲcallsto publicmethodsandreadsorwritesofpublicfieldsͲshouldbeabletoviolateanyofitsinvariantsorpostͲconditions.No setofoperationsperformedsequentiallyorconcurrentlyoninstancesofathreadͲsafeclasscancauseaninstancetobe inaninvalidstate. [2]Ifthelooseuseof"correctness"herebothersyou,youmayprefertothinkofathreadͲsafeclassasonethatisnomorebrokeninaconcurrent environmentthaninasingleͲthreadedenvironment.  13 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety ThreadͲsafeclassesencapsulateanyneededsynchronizationsothatclientsneednotprovidetheirown. 2.1.1.Example:AStatelessServlet InChapter1,welistedanumberofframeworksthatcreatethreadsandcallyourcomponentsfromthosethreads, leaving you with the responsibility of making your components threadͲsafe. Very often, threadͲsafety requirements stemnotfromadecisiontousethreadsdirectlybutfromadecisiontouseafacilityliketheServletsframework.We're goingtodevelopasimpleexampleͲaservletͲbasedfactorizationserviceͲandslowlyextendittoaddfeatureswhile preservingitsthreadsafety. Listing2.1showsoursimplefactorizationservlet.Itunpacksthenumbertobefactoredfromtheservletrequest,factors it,andpackagestheresultsintotheservletresponse. Listing2.1.AStatelessServlet. @ThreadSafe public class StatelessFactorizer implements Servlet { public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = factor(i); encodeIntoResponse(resp, factors); } } StatelessFactorizeris,likemostservlets,stateless:ithasnofieldsandreferencesnofieldsfromotherclasses.The transientstateforaparticularcomputationexistssolelyinlocalvariablesthatarestoredonthethread'sstackandare accessibleonlytotheexecutingthread.OnethreadaccessingaStatelessFactorizercannotinfluencetheresultof anotherthreadaccessingthesameStatelessFactorizer;becausethetwothreadsdonotsharestate,itisasifthey were accessing different instances. Since the actions of a thread accessing a stateless object cannot affect the correctnessofoperationsinotherthreads,statelessobjectsarethreadͲsafe. StatelessobjectsarealwaysthreadͲsafe.  ThefactthatmostservletscanbeimplementedwithnostategreatlyreducestheburdenofmakingservletsthreadͲ safe.Itisonlywhenservletswanttorememberthingsfromonerequesttoanotherthatthethreadsafetyrequirement becomesanissue. 2.2.Atomicity Whathappenswhenwe addoneelementofstatetowhatwasastatelessobject?Supposewe wanttoadda"hit counter"thatmeasuresthenumberofrequestsprocessed.Theobviousapproachistoaddalongfieldtotheservlet andincrementitoneachrequest,asshowninUnsafeCountingFactorizerinListing2.2. Unfortunately,UnsafeCountingFactorizerisnotthreadͲsafe,eventhoughitwouldworkjustfineinasingleͲthreaded environment. Just like UnsafeSequence on page 6, it is susceptible to lost updates. While the increment operation, ++count,maylooklikeasingleactionbecauseofitscompactsyntax,itisnotatomic,whichmeansthatitdoesnot executeasasingle,indivisibleoperation.Instead,itisshorthandforasequenceofthreediscreteoperations:fetchthe currentvalue,addonetoit,andwritethenewvalueback.ThisisanexampleofareadͲmodifyͲwriteoperation,inwhich theresultingstateisderivedfromthepreviousstate.   14 JavaConcurrencyInPractice Listing2.2.ServletthatCountsRequestswithouttheNecessarySynchronization.Don'tDothis. @NotThreadSafe public class UnsafeCountingFactorizer implements Servlet { private long count = 0; public long getCount() { return count; } public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = factor(i); ++count; encodeIntoResponse(resp, factors); } } Figure 1.1 on page 6 shows what can happen if two threads try to increment a counter simultaneously without synchronization.Ifthecounterisinitially9,withsomeunluckytimingeachthreadcouldreadthevalue,seethatitis9, addonetoit,andeachsetthecounterto10.Thisisclearlynotwhatissupposedtohappen;anincrementgotlostalong theway,andthehitcounterisnowpermanentlyoffbyone. YoumightthinkthathavingaslightlyinaccuratecountofhitsinawebͲbasedserviceisanacceptablelossofaccuracy, andsometimesitis.Butifthecounterisbeingusedtogeneratesequencesoruniqueobjectidentifiers,returningthe samevaluefrommultipleinvocationscouldcauseseriousdataintegrityproblems.[3]Thepossibilityofincorrectresults inthepresenceofunluckytimingissoimportantinconcurrentprogrammingthatithasaname:aracecondition. [3] The approach taken by UnsafeSequence and UnsafeCountingFactorizer has other serious problems, including the possibility of stale data (Section3.1.1). 2.2.1.RaceConditions UnsafeCountingFactorizerhasseveralraceconditionsthatmakeitsresultsunreliable.Araceconditionoccurswhen thecorrectnessofacomputationdependsontherelativetimingorinterleavingofmultiplethreadsbytheruntime;in otherwords,whengettingtherightanswerreliesonluckytiming.[4]ThemostcommontypeofraceconditionischeckͲ thenͲact,whereapotentiallystaleobservationisusedtomakeadecisiononwhattodonext. [4]Thetermraceconditionisoftenconfusedwiththerelatedtermdatarace,whichariseswhensynchronizationisnotusedtocoordinateall accesstoasharednonͲfinalfield.Youriskadataracewheneverathreadwritesavariablethatmightnextbereadbyanotherthreadorreadsa variablethatmighthavelastbeenwrittenbyanotherthreadifboththreadsdonotusesynchronization;codewithdataraceshasnouseful definedsemanticsundertheJavaMemoryModel.Notallraceconditionsaredataraces,andnotalldataracesareraceconditions,buttheyboth cancauseconcurrentprogramstofailinunpredictableways.UnsafeCountingFactorizerhasbothraceconditionsanddataraces.SeeChapter 16formoreondataraces. Weoftenencounterraceconditionsinreallife.Let'ssayyouplannedtomeetafriendatnoonattheStarbuckson UniversityAvenue.Butwhenyougetthere,yourealizetherearetwoStarbucksonUniversityAvenue,andyou'renot surewhichoneyouagreedtomeetat.At12:10,youdon'tseeyourfriendatStarbucksA,soyouwalkovertoStarbucks B to see if he's there, but he isn't there either. There are a few possibilities: your friend is late and not at either Starbucks;yourfriendarrivedatStarbucksAafteryouleft;oryourfriendwasatStarbucksB,butwenttolookforyou, andisnowenroutetoStarbucksA.Let'sassumetheworstandsayitwasthelastpossibility.Nowit's12:15,you'veboth beentobothStarbucks,andyou'rebothwonderingifyou'vebeenstoodup.Whatdoyoudonow?Gobacktotheother Starbucks?Howmanytimesareyougoingtogobackandforth?Unlessyouhaveagreedonaprotocol,youcouldboth spendthedaywalkingupanddownUniversityAvenue,frustratedandundercaffeinated. Theproblemwiththe"I'lljustnipupthestreetandseeifhe'sattheotherone"approachisthatwhileyou'rewalkingup thestreet,yourfriendmighthavemoved.YoulookaroundStarbucksA,observe"he'snothere",andgolookingforhim. AndyoucandothesameforStarbucksB,butnotatthesametime.Ittakesafewminutestowalkupthestreet,and duringthosefewminutes,thestateofthesystemmayhavechanged. TheStarbucksexampleillustratesaraceconditionbecausereachingthedesiredoutcome(meetingyourfriend)depends ontherelativetimingofevents(wheneachofyouarrivesatoneStarbucksortheother,howlongyouwaittherebefore switching,etc).TheobservationthatheisnotatStarbucksAbecomespotentiallyinvalidassoonasyouwalkoutthe frontdoor;hecouldhavecomeinthroughthebackdoorandyouwouldn'tknow.Itisthisinvalidationofobservations that characterizes most race conditionsͲusing a potentially stale observation to make a decision or perform a computation.ThistypeofraceconditioniscalledcheckͲthenͲact:youobservesomethingtobetrue(fileXdoesn'texist)  15 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety and then take action based on that observation (create X); but in fact the observation could have become invalid betweenthetimeyouobserveditandthetimeyouactedonit(someoneelsecreatedXinthemeantime),causinga problem(unexpectedexception,overwrittendata,filecorruption). 2.2.2.Example:RaceConditionsinLazyInitialization AcommonidiomthatusescheckͲthenͲactislazyinitialization.Thegoaloflazyinitializationistodeferinitializingan objectuntilitisactuallyneededwhileatthesametimeensuringthatitisinitializedonlyonce.LazyInitRaceinListing 2.3illustratesthelazyinitializationidiom.The getInstance methodfirst checkswhetherthe ExpensiveObject has alreadybeeninitialized,inwhichcaseitreturnstheexistinginstance;otherwiseitcreatesanewinstanceandreturnsit afterretainingareferencetoitsothatfutureinvocationscanavoidthemoreexpensivecodepath. Listing2.3.RaceConditioninLazyInitialization.Don'tDothis. @NotThreadSafe public class LazyInitRace { private ExpensiveObject instance = null; public ExpensiveObject getInstance() { if (instance == null) instance = new ExpensiveObject(); return instance; } } LazyInitRacehasraceconditionsthatcanundermineitscorrectness.SaythatthreadsAandBexecutegetInstanceat thesametime.Aseesthatinstanceisnull,andinstantiatesanewExpensiveObject.Balsochecksifinstanceis null.Whetherinstanceisnullatthispointdependsunpredictablyontiming,includingthevagariesofschedulingand howlongAtakestoinstantiatetheExpensiveObjectandsettheinstancefield.IfinstanceisnullwhenBexamines it,thetwocallerstogetInstancemayreceivetwodifferentresults,eventhoughgetInstanceisalwayssupposedto returnthesameinstance. The hitͲcounting operation in UnsafeCountingFactorizer has another sort of race condition. ReadͲmodifyͲwrite operations,likeincrementingacounter,defineatransformationofanobject'sstateintermsofitspreviousstate.To incrementacounter,youhavetoknowitspreviousvalueandmakesurenooneelsechangesorusesthatvaluewhile youareinmidͲupdate. Likemostconcurrencyerrors,raceconditionsdon'talwaysresultinfailure:someunluckytimingisalsorequired.But raceconditionscancauseseriousproblems.IfLazyInitRaceisusedtoinstantiateanapplicationͲwideregistry,havingit returndifferentinstancesfrommultipleinvocationscouldcauseregistrationstobelostormultipleactivitiestohave inconsistent views of the set of registered objects. If UnsafeSequence is used to generate entity identifiers in a persistenceframework,twodistinctobjectscouldendupwiththesameID,violatingidentityintegrityconstraints. 2.2.3.CompoundActions BothLazyInitRaceandUnsafeCountingFactorizercontainedasequenceofoperationsthatneededtobeatomic,or indivisible,relativetootheroperationsonthesamestate.Toavoidraceconditions,theremustbeawaytoprevent otherthreadsfromusingavariablewhilewe'reinthemiddleofmodifyingit,sowecanensurethatotherthreadscan observeormodifythestateonlybeforewestartorafterwefinish,butnotinthemiddle. Operations A and B are atomic with respect to each other if, from the perspective of a thread executing A, when anotherthreadexecutesB,eitherallofBhasexecutedornoneofithas.Anatomicoperationisonethatisatomicwith respecttoalloperations,includingitself,thatoperateonthesamestate.  IftheincrementoperationinUnsafeSequencewereatomic,theraceconditionillustratedinFigure1.1onpage6could notoccur,andeachexecutionoftheincrementoperationwouldhavethedesiredeffectofincrementingthecounterby exactly one. To ensure thread safety, checkͲthenͲact operations (like lazy initialization) and readͲmodifyͲwrite operations (like increment) must always be atomic. We refer collectively to checkͲthenͲact and readͲmodifyͲwrite sequencesascompoundactions:sequencesofoperationsthatmustbeexecutedatomicallyinordertoremainthreadͲ  16 JavaConcurrencyInPractice safe.Inthenextsection,we'llconsiderlocking,Java'sbuiltͲinmechanismforensuringatomicity.Fornow,we'regoingto fixtheproblemanotherway,byusinganexistingthreadͲsafeclass,asshowninCountingFactorizerinListing2.4. Listing2.4.ServletthatCountsRequestsUsingAtomicLong. @ThreadSafe public class CountingFactorizer implements Servlet { private final AtomicLong count = new AtomicLong(0); public long getCount() { return count.get(); } public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = factor(i); count.incrementAndGet(); encodeIntoResponse(resp, factors); } } Thejava.util.concurrent.atomicpackagecontainsatomicvariableclassesforeffectingatomicstatetransitionson numbersandobjectreferences.Byreplacingthe longcounter withan AtomicLong,we ensurethat allactionsthat accessthecounterstateareatomic.[5]Becausethestateoftheservletisthestateofthecounterandthecounteris threadͲsafe,ourservletisonceagainthreadͲsafe. [5]CountingFactorizercallsincrementAndGettoincrementthecounter,whichalsoreturnstheincrementedvalue;inthiscasethereturnvalueis ignored. WewereabletoaddacountertoourfactoringservletandmaintainthreadsafetybyusinganexistingthreadͲsafeclass tomanagethecounterstate,AtomicLong.Whenasingleelementofstateisaddedtoastatelessclass,theresulting classwillbethreadͲsafeifthestateisentirelymanagedbyathreadͲsafeobject.But,aswe'llseeinthenextsection, goingfromonestatevariabletomorethanoneisnotnecessarilyassimpleasgoingfromzerotoone. Wherepractical,useexistingthreadͲsafeobjects,likeAtomicLong,tomanageyourclass'sstate.Itissimplertoreason aboutthepossiblestatesandstatetransitionsforexistingthreadͲsafeobjectsthanitisforarbitrarystatevariables,and thismakesiteasiertomaintainandverifythreadsafety. 2.3.Locking WewereabletoaddonestatevariabletoourservletwhilemaintainingthreadsafetybyusingathreadͲsafeobjectto managetheentirestateoftheservlet.Butifwewanttoaddmorestatetoourservlet,canwejustaddmorethreadͲ safestatevariables? Imaginethatwewanttoimprovetheperformanceofourservletbycachingthemostrecentlycomputedresult,justin case two consecutive clients request factorization of the same number. (This is unlikely to be an effective caching strategy;weofferabetteroneinSection5.6.)Toimplementthisstrategy,weneedtoremembertwothings:thelast numberfactored,anditsfactors. We used AtomicLong to manage the counter state in a threadͲsafe manner; could we perhaps use its cousin, AtomicReference, [6] to manage the last number and its factors? An attempt at this is shown in UnsafeCachingFactorizerinListing2.5. [6]JustasAtomicLongisathreadͲsafeholderclassforalonginteger,AtomicReferenceisathreadsafeholderclassforanobjectreference.Atomic variablesandtheirbenefitsarecoveredinChapter15.  17 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety Listing2.5.ServletthatAttemptstoCacheitsLastResultwithoutAdequateAtomicity.Don'tDothis. @NotThreadSafe public class UnsafeCachingFactorizer implements Servlet { private final AtomicReference lastNumber = new AtomicReference(); private final AtomicReference lastFactors = new AtomicReference(); public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); if (i.equals(lastNumber.get())) encodeIntoResponse(resp, lastFactors.get() ); else { BigInteger[] factors = factor(i); lastNumber.set(i); lastFactors.set(factors); encodeIntoResponse(resp, factors); } } } Unfortunately, this approach does not work. Even though the atomic references are individually threadͲsafe, UnsafeCachingFactorizerhasraceconditionsthatcouldmakeitproducethewronganswer. Thedefinitionofthreadsafetyrequiresthatinvariantsbepreservedregardlessoftimingorinterleavingofoperationsin multiplethreads.OneinvariantofUnsafeCachingFactorizeristhattheproductofthefactorscachedinlastFactors equalthevaluecachedinlastNumber;ourservletiscorrectonlyifthisinvariantalwaysholds.Whenmultiplevariables participateinaninvariant,theyarenotindependent:thevalueofoneconstrainstheallowedvalue(s)oftheothers. Thuswhenupdatingone,youmustupdatetheothersinthesameatomicoperation. Withsomeunluckytiming,UnsafeCachingFactorizercanviolatethisinvariant.Usingatomicreferences,wecannot update both lastNumber and lastFactors simultaneously, even though each call to set is atomic; there is still a windowofvulnerabilitywhenonehasbeenmodifiedandtheotherhasnot,andduringthattimeotherthreadscould seethattheinvariantdoesnothold.Similarly,thetwovaluescannotbefetchedsimultaneously:betweenthetime whenthreadAfetchesthetwovalues,threadBcouldhavechangedthem,andagainAmayobservethattheinvariant doesnothold. Topreservestateconsistency,updaterelatedstatevariablesinasingleatomicoperation. 2.3.1.IntrinsicLocks Java provides a builtͲin locking mechanism for enforcing atomicity: the synchronized block. (There is also another critical aspect to locking and other synchronization mechanismsvisibilityͲwhich is covered in Chapter 3.) A synchronized block has two parts: a reference to an object that will serve as the lock, and a block of code to be guardedbythatlock.A synchronizedmethodisshorthandfora synchronizedblockthatspansanentiremethod body,andwhoselockistheobjectonwhichthemethodisbeinginvoked.(StaticsynchronizedmethodsusetheClass objectforthelock.) synchronized (lock) { // Access or modify shared state guarded by lock } EveryJavaobjectcanimplicitlyactasalockforpurposesofsynchronization;thesebuiltͲinlocksarecalledintrinsiclocks ormonitorlocks.Thelockisautomaticallyacquiredbytheexecutingthreadbeforeenteringasynchronizedblockand automaticallyreleasedwhencontrolexitsthesynchronizedblock,whetherbythenormalcontrolpathorbythrowing anexceptionoutoftheblock.Theonlywaytoacquireanintrinsiclockistoenterasynchronizedblockormethod guardedbythatlock. IntrinsiclocksinJavaactasmutexes(ormutualexclusionlocks),whichmeansthatatmostonethreadmayownthe lock.WhenthreadAattemptstoacquirealockheldbythreadB,Amustwait,orblock,untilBreleasesit.IfBnever releasesthelock,Awaitsforever.  18 JavaConcurrencyInPractice Sinceonlyonethreadatatimecanexecuteablockofcodeguardedbyagivenlock,thesynchronizedblocksguarded bythesamelockexecuteatomicallywithrespecttooneanother.Inthecontextofconcurrency,atomicitymeansthe samethingasitdoesintransactionalapplicationsͲthatagroupofstatementsappeartoexecuteasasingle,indivisible unit.Nothreadexecutingasynchronizedblockcanobserveanotherthreadtobeinthemiddleofasynchronized blockguardedbythesamelock. Themachineryofsynchronizationmakesiteasytorestorethreadsafetytothefactoringservlet.Listing2.6makesthe servicemethod synchronized,soonlyonethreadmayenter serviceatatime.SynchronizedFactorizerisnow threadͲsafe;however,thisapproachisfairlyextreme,sinceitinhibitsmultipleclientsfromusingthefactoringservlet simultaneouslyatallͲresultinginunacceptablypoorresponsiveness.ThisproblemͲwhichisaperformanceproblem, notathreadsafetyproblemͲisaddressedinSection2.5. Listing2.6.ServletthatCachesLastResult,ButwithUnacceptablyPoorConcurrency.Don'tDothis. @ThreadSafe public class SynchronizedFactorizer implements Servlet { @GuardedBy("this") private BigInteger lastNumber; @GuardedBy("this") private BigInteger[] lastFactors; public synchronized void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); if (i.equals(lastNumber)) encodeIntoResponse(resp, lastFactors); else { BigInteger[] factors = factor(i); lastNumber = i; lastFactors = factors; encodeIntoResponse(resp, factors); } } } 2.3.2.Reentrancy When a thread requests a lock that is already held by another thread, the requesting thread blocks. But because intrinsiclocksarereentrant,ifathreadtriestoacquirealockthatitalreadyholds,therequestsucceeds.Reentrancy means that locks are acquired on a perͲthread rather than perͲinvocation basis. [7] Reentrancy is implemented by associatingwitheachlockanacquisitioncountandanowningthread.Whenthecountiszero,thelockisconsidered unheld.Whenathreadacquiresapreviouslyunheldlock,theJVMrecordstheownerandsetstheacquisitioncountto one. If that same thread acquires the lock again, the count is incremented, and when the owning thread exits the synchronizedblock,thecountisdecremented.Whenthecountreacheszero,thelockisreleased. [7]Thisdiffersfromthedefaultlockingbehaviorforpthreads(POSIXthreads)mutexes,whicharegrantedonaperͲinvocationbasis. Reentrancy facilitates encapsulation of locking behavior, and thus simplifies the development of objectͲoriented concurrentcode.Withoutreentrantlocks,theverynaturalͲlookingcodeinListing2.7,inwhichasubclassoverridesa synchronizedmethodandthencallsthesuperclassmethod,woulddeadlock.BecausethedoSomethingmethodsin WidgetandLoggingWidgetarebothsynchronized,eachtriestoacquirethelockontheWidgetbeforeproceeding. Butifintrinsiclockswerenotreentrant,thecalltosuper.doSomethingwouldneverbeabletoacquirethelockbecause itwouldbeconsideredalreadyheld,andthethreadwouldpermanentlystallwaitingforalockitcanneveracquire. Reentrancysavesusfromdeadlockinsituationslikethis. Listing2.7.CodethatwouldDeadlockifIntrinsicLockswereNotReentrant. public class Widget { public synchronized void doSomething() { ... } } public class LoggingWidget extends Widget { public synchronized void doSomething() { System.out.println(toString() + ": calling doSomething"); super.doSomething(); } }  19 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety 2.4.GuardingStatewithLocks Becauselocksenableserialized [8]accesstothecodepathstheyguard,wecanusethemtoconstructprotocolsfor guaranteeingexclusiveaccesstosharedstate.Followingtheseprotocolsconsistentlycanensurestateconsistency. [8]Serializingaccesstoanobjecthasnothingtodowithobjectserialization(turninganobjectintoabytestream);serializingaccessmeansthat threadstaketurnsaccessingtheobjectexclusively,ratherthandoingsoconcurrently. Compoundactionsonsharedstate,suchasincrementingahitcounter(readͲmodifyͲwrite)orlazyinitialization(checkͲ thenͲact),mustbemadeatomictoavoidraceconditions.Holdingalockfortheentiredurationofacompoundaction canmakethatcompoundactionatomic.However,justwrappingthecompoundactionwithasynchronizedblockisnot sufficient;ifsynchronizationisusedtocoordinateaccesstoavariable,itisneededeverywherethatvariableisaccessed. Further,whenusinglockstocoordinateaccesstoavariable,thesamelockmustbeusedwhereverthatvariableis accessed. Itisacommonmistaketoassumethatsynchronizationneedstobeusedonlywhenwritingtosharedvariables;thisis simplynottrue.(ThereasonsforthiswillbecomeclearerinSection3.1.) Foreachmutablestatevariablethatmaybeaccessedbymorethanonethread,allaccessestothatvariablemustbe performedwiththesamelockheld.Inthiscase,wesaythatthevariableisguardedbythatlock. InSynchronizedFactorizerinListing2.6,lastNumberandlastFactorsareguardedbytheservletobject'sintrinsic lock;thisisdocumentedbythe@GuardedByannotation. Thereisnoinherentrelationshipbetweenanobject'sintrinsiclockanditsstate;anobject'sfieldsneednotbeguarded byitsintrinsiclock,thoughthisisaperfectlyvalidlockingconventionthatisusedbymanyclasses.Acquiringthelock associatedwithanobjectdoesnotpreventotherthreadsfromaccessingthatobjectͲtheonlythingthatacquiringa lockpreventsanyotherthreadfromdoingisacquiringthatsamelock.ThefactthateveryobjecthasabuiltͲinlockis justaconveniencesothatyouneedn'texplicitlycreatelockobjects.[9]Itisuptoyoutoconstructlockingprotocolsor synchronizationpoliciesthatletyouaccesssharedstatesafely,andtousethemconsistentlythroughoutyourprogram. [9]Inretrospect,thisdesigndecisionwasprobablyabadone:notonlycanitbeconfusing,butitforcesJVMimplementerstomaketradeoffs betweenobjectsizeandlockingperformance. Everyshared,mutablevariableshouldbeguardedbyexactlyonelock.Makeitcleartomaintainerswhichlockthatis. Acommonlockingconventionistoencapsulateallmutablestatewithinanobjectandtoprotectitfromconcurrent accessbysynchronizinganycodepaththataccessesmutablestateusingtheobject'sintrinsiclock.Thispatternisused bymanythreadͲsafeclasses,suchasVectorandothersynchronizedcollectionclasses.Insuchcases,allthevariablesin anobject'sstateareguardedbytheobject'sintrinsiclock.However,thereisnothingspecialaboutthispattern,and neitherthecompilernortheruntimeenforcesthis(oranyother)patternoflocking. [10]Itisalsoeasytosubvertthis lockingprotocolaccidentallybyaddinganewmethodorcodepathandforgettingtousesynchronization. [10]CodeauditingtoolslikeFindBugscanidentifywhenavariableisfrequentlybutnotalwaysaccessedwithalockheld,whichmayindicatea bug. NotalldataneedstobeguardedbylocksͲonlymutabledatathatwillbeaccessedfrommultiplethreads.InChapter1, wedescribedhowaddingasimpleasynchronouseventsuchasaTimerTaskcancreatethreadsafetyrequirementsthat ripplethroughoutyourprogram,especiallyifyourprogramstateispoorlyencapsulated.ConsiderasingleͲthreaded programthatprocessesalargeamountofdata.SingleͲthreadedprogramsrequirenosynchronization,becausenodata issharedacrossthreads.Nowimagineyouwanttoaddafeaturetocreateperiodicsnapshotsofitsprogress,sothatit doesnothavetostartagainfromthebeginningifitcrashesormustbestopped.Youmightchoosetodothiswitha TimerTaskthatgoesoffeverytenminutes,savingtheprogramstatetoafile. SincetheTimerTaskwillbecalledfromanotherthread(onemanagedbyTimer),anydatainvolvedinthesnapshotis now accessed by two threads: the main program thread and the Timer thread. This means that not only must the TimerTaskcodeusesynchronizationwhenaccessingtheprogramstate,butsomustanycodepathintherestofthe program that touches that same data. What used to require no synchronization now requires synchronization throughouttheprogram.  20 JavaConcurrencyInPractice WhenavariableisguardedbyalockͲmeaningthateveryaccesstothatvariableisperformedwiththatlockheldͲ you'veensuredthatonlyonethreadatatimecanaccessthatvariable.Whenaclasshasinvariantsthatinvolvemore thanonestatevariable,thereisanadditionalrequirement:eachvariableparticipatingintheinvariantmustbeguarded by the same lock. This allows you to access or update them in a single atomic operation, preserving the invariant. SynchronizedFactorizerdemonstratesthisrule:boththecachednumberandthecachedfactorsareguardedbythe servletobject'sintrinsiclock. Foreveryinvariantthatinvolvesmorethanonevariable,allthevariablesinvolvedinthatinvariantmustbeguardedby thesamelock. Ifsynchronizationisthecureforraceconditions,whynotjustdeclareeverymethodsynchronized?Itturnsoutthat such indiscriminate application of synchronized might be either too much or too little synchronization. Merely synchronizingeverymethod,asVectordoes,isnotenoughtorendercompoundactionsonaVectoratomic: if (!vector.contains(element)) vector.add(element); ThisattemptataputͲifͲabsentoperationhasaracecondition,eventhoughbothcontainsandaddareatomic.While synchronizedmethodscanmakeindividualoperationsatomic,additionallockingisrequiredͲwhenmultipleoperations are combined into a compound action. (See Section 4.4 for some techniques for safely adding additional atomic operationstothreadͲsafeobjects.)Atthesametime,synchronizingeverymethodcanleadtolivenessorperformance problems,aswesawinSynchronizedFactorizer. 2.5.LivenessandPerformance In UnsafeCachingFactorizer, we introduced some caching into our factoring servlet in the hope of improving performance.Cachingrequiredsomesharedstate,whichinturnrequiredsynchronizationtomaintaintheintegrityof that state. But the way we used synchronization in SynchronizedFactorizer makes it perform badly. The synchronizationpolicyforSynchronizedFactorizeristoguardeachstatevariablewiththeservletobject'sintrinsic lock, and that policy was implemented by synchronizing the entirety of the service method. This simple, coarseͲ grainedapproachrestoredsafety,butatahighprice. Becauseserviceissynchronized,onlyonethreadmayexecuteitatonce.Thissubvertstheintendeduseoftheservlet frameworkͲthatservletsbeabletohandlemultiplerequestssimultaneouslyͲandcanresultinfrustratedusersifthe loadishighenough.Iftheservletisbusyfactoringalargenumber,otherclientshavetowaituntilthecurrentrequestis completebeforetheservletcanstartonthenewnumber.IfthesystemhasmultipleCPUs,processorsmayremainidle eveniftheloadishigh.Inanycase,evenshortͲrunningrequests,suchasthoseforwhichthevalueiscached,maytake anunexpectedlylongtimebecausetheymustwaitforpreviouslongͲrunningrequeststocomplete. Figure2.1showswhathappenswhenmultiplerequestsarriveforthesynchronizedfactoringservlet:theyqueueupand are handled sequentially. We would describe this web application as exhibiting poor concurrency: the number of simultaneous invocations is limited not by the availability of processing resources, but by the structure of the applicationitself.Fortunately,itiseasytoimprovetheconcurrencyoftheservletwhilemaintainingthreadsafetyby narrowingthescopeofthesynchronizedblock.Youshouldbecarefulnottomakethescopeofthesynchronized blocktoosmall;youwouldnotwanttodivideanoperationthatshouldbeatomicintomorethanonesynchronized block.ButitisreasonabletotrytoexcludefromsynchronizedblockslongͲrunningoperationsthatdonotaffectshared state,sothatotherthreadsarenotpreventedfromaccessingthesharedstatewhilethelongͲrunningoperationisin progress.  21 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ14BChapter2.Thread Safety Figure2.1.PoorConcurrencyofSynchronizedFactorizer. CachedFactorizerinListing2.8restructurestheservlettousetwoseparatesynchronizedblocks,eachlimitedtoa shortsectionofcode.OneguardsthecheckͲthenͲactsequencethattestswhetherwecanjustreturnthecachedresult, andtheotherguardsupdatingboththecachednumberandthecachedfactors.Asabonus,we'vereintroducedthehit counterandaddeda"cachehit"counteraswell,updatingthemwithintheinitialsynchronizedblock.Becausethese counters constitute shared mutable state as well, we must use synchronization everywhere they are accessed. The portionsofcodethatareoutsidethesynchronizedblocksoperateexclusivelyonlocal(stackͲbased)variables,which arenotsharedacrossthreadsandthereforedonotrequiresynchronization. Listing2.8.ServletthatCachesitsLastRequestandResult. @ThreadSafe public class CachedFactorizer implements Servlet { @GuardedBy("this") private BigInteger lastNumber; @GuardedBy("this") private BigInteger[] lastFactors; @GuardedBy("this") private long hits; @GuardedBy("this") private long cacheHits; public synchronized long getHits() { return hits; } public synchronized double getCacheHitRatio() { return (double) cacheHits / (double) hits; } public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = null; synchronized (this) { ++hits; if (i.equals(lastNumber)) { ++cacheHits; factors = lastFactors.clone(); } } if (factors == null) { factors = factor(i); synchronized (this) { lastNumber = i; lastFactors = factors.clone(); } } encodeIntoResponse(resp, factors); } } CachedFactorizernolongerusesAtomicLongforthehitcounter,insteadrevertingtousingalongfield.Itwouldbe safetouse AtomicLonghere,butthereislessbenefitthantherewasinCountingFactorizer.Atomicvariablesare useful for effecting atomic operations on a single variable, but since we are already using synchronized blocks to constructatomicoperations,usingtwodifferentsynchronizationmechanismswouldbeconfusingandwouldofferno performanceorsafetybenefit. TherestructuringofCachedFactorizerprovidesabalancebetweensimplicity(synchronizingtheentiremethod)and concurrency(synchronizingtheshortestpossiblecodepaths).Acquiringandreleasingalockhassomeoverhead,soitis undesirabletobreakdownsynchronizedblockstoofar(suchasfactoring++hitsintoitsownsynchronizedblock), evenifthiswouldnotcompromiseatomicity.CachedFactorizerholdsthelockwhenaccessingstatevariablesandfor thedurationofcompoundactions,butreleasesitbeforeexecutingthepotentiallylongͲrunningfactorizationoperation.  22 JavaConcurrencyInPractice Thispreservesthreadsafetywithoutundulyaffectingconcurrency;thecodepathsineachofthesynchronizedblocks are"shortenough". Deciding how big or small to make synchronized blocks may require tradeoffs among competing design forces, includingsafety(whichmustnotbecompromised),simplicity,andperformance.Sometimessimplicityandperformance areatoddswitheachother,althoughasCachedFactorizerillustrates,areasonablebalancecanusuallybefound. Thereisfrequentlyatensionbetweensimplicityandperformance.Whenimplementingasynchronizationpolicy,resist thetemptationtoprematurelysacrificesimplicity(potentiallycompromisingsafety)forthesakeofperformance. Wheneveryouuselocking,youshouldbeawareofwhatthecodeintheblockisdoingandhowlikelyitistotakealong timetoexecute.Holdingalockforalongtime,eitherbecauseyouaredoingsomethingcomputeͲintensiveorbecause youexecuteapotentiallyblockingoperation,introducestheriskoflivenessorperformanceproblems. Avoidholdinglocksduringlengthycomputationsoroperationsatriskofnotcompletingquicklysuchasnetworkor consoleI/O.  23 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects Chapter3.SharingObjects WestatedatthebeginningofChapter2thatwritingcorrectconcurrentprogramsisprimarilyaboutmanagingaccessto shared,mutablestate.Thatchapterwasaboutusingsynchronizationtopreventmultiplethreadsfromaccessingthe samedataatthesametime;thischapterexaminestechniquesforsharingandpublishingobjectssotheycanbesafely accessedbymultiplethreads.Together,theylaythefoundationforbuildingthreadͲsafeclassesandsafelystructuring concurrentapplicationsusingthejava.util.concurrentlibraryclasses. We have seen how synchronized blocks and methods can ensure that operations execute atomically, but it is a commonmisconceptionthatsynchronizedisonlyaboutatomicityordemarcating"criticalsections".Synchronization also has another significant, and subtle, aspect: memory visibility. We want not only to prevent one thread from modifyingthestateofanobjectwhenanotherisusingit,butalsotoensurethatwhenathreadmodifiesthestateofan object,otherthreadscanactuallyseethechangesthatweremade.Butwithoutsynchronization,thismaynothappen. Youcanensurethatobjectsarepublishedsafelyeitherbyusingexplicitsynchronizationorbytakingadvantageofthe synchronizationbuiltintolibraryclasses. 3.1.Visibility Visibilityissubtlebecausethethingsthatcangowrongaresocounterintuitive.InasingleͲthreadedenvironment,ifyou writeavaluetoavariableandlaterreadthatvariablewithnointerveningwrites,youcanexpecttogetthesamevalue back.Thisseemsonlynatural.Itmaybehardtoacceptatfirst,butwhenthereadsandwritesoccurindifferentthreads, thisissimplynotthecase.Ingeneral,thereisnoguaranteethatthereadingthreadwillseeavaluewrittenbyanother threadonatimelybasis,orevenatall.Inordertoensurevisibilityofmemorywritesacrossthreads,youmustuse synchronization. NoVisibility in Listing 3.1 illustrates what can go wrong when threads share data without synchronization. Two threads,themainthreadandthereaderthread,accessthesharedvariablesreadyandnumber.Themainthreadstarts thereaderthreadandthensetsnumberto42andreadytotrue.Thereaderthreadspinsuntilitseesreadyistrue,and thenprintsoutnumber.WhileitmayseemobviousthatNoVisibilitywillprint42,itisinfactpossiblethatitwillprint zero,orneverterminateatall!Becauseitdoesnotuseadequatesynchronization,thereisnoguaranteethatthevalues ofreadyandnumberwrittenbythemainthreadwillbevisibletothereaderthread. Listing3.1.SharingVariableswithoutSynchronization.Don'tDothis. public class NoVisibility { private static boolean ready; private static int number; private static class ReaderThread extends Thread { public void run() { while (!ready) Thread.yield(); System.out.println(number); } } public static void main(String[] args) { new ReaderThread().start(); number = 42; ready = true; } } NoVisibilitycouldloopforeverbecausethevalueofreadymightneverbecomevisibletothereaderthread.Even morestrangely,NoVisibilitycouldprintzerobecausethewritetoreadymightbemadevisibletothereaderthread beforethewritetonumber,aphenomenonknownasreordering.Thereisnoguaranteethatoperationsinonethread willbeperformedintheordergivenbytheprogram,aslongasthereorderingisnotdetectablefromwithinthatthread Ͳevenifthereorderingisapparenttootherthreads.[1]Whenthemainthreadwritesfirsttonumberandthentodone withoutsynchronization,thereaderthreadcouldseethosewriteshappenintheoppositeorderͲornotatall. [1] This may seem like a broken design, but it is meant to allow JVMs to take full advantage of the performance of modern multiprocessor hardware.Forexample,intheabsenceofsynchronization,theJavaMemoryModelpermitsthecompilertoreorderoperationsandcachevaluesin registers,andpermitsCPUstoreorderoperationsandcachevaluesinprocessorͲspecificcaches.Formoredetails,seeChapter16.  24 JavaConcurrencyInPractice Intheabsenceofsynchronization,thecompiler,processor,andruntimecandosomedownrightweirdthingstothe orderinwhichoperationsappeartoexecute.Attemptstoreasonabouttheorderinwhichmemoryactions"must" happenininsufficientlysynchronizedmultithreadedprogramswillalmostcertainlybeincorrect. NoVisibilityisaboutassimpleasaconcurrentprogramcangetͲtwothreadsandtwosharedvariablesͲandyetitis stillalltooeasytocometothewrongconclusionsaboutwhatitdoesorevenwhetheritwillterminate.Reasoning aboutinsufficientlysynchronizedconcurrentprogramsisprohibitivelydifficult. Thismayallsoundalittlescary,anditshould.Fortunately,there'saneasywaytoavoidthesecomplexissues:always usethepropersynchronizationwheneverdataissharedacrossthreads. 3.1.1.StaleData NoVisibilitydemonstratedoneofthewaysthatinsufficientlysynchronizedprogramscancausesurprisingresults: staledata.Whenthereaderthreadexaminesready,itmayseeanoutͲofͲdatevalue.Unlesssynchronizationisused every time a variable is accessed, it is possible tosee a stale value for that variable. Worse, staleness is not allͲorͲ nothing:athreadcanseeanupͲtoͲdatevalueofonevariablebutastalevalueofanothervariablethatwaswrittenfirst. Whenfoodisstale,itisusuallystilledibleͲjustlessenjoyable.Butstaledatacanbemoredangerous.WhileanoutͲofͲ datehitcounterinawebapplicationmightnotbesobad,[2]stalevaluescancauseserioussafetyorlivenessfailures.In NoVisibility,stalevaluescouldcauseittoprintthewrongvalueorpreventtheprogramfromterminating.Thingscan get even more complicated with stale values of object references, such as the link pointers in a linked list implementation.Staledatacan cause seriousand confusingfailuressuchasunexpected exceptions,corrupted data structures,inaccuratecomputations,andinfiniteloops. [2]ReadingdatawithoutsynchronizationisanalogoustousingtheREAD_UNCOMMITTEDisolationlevelinadatabase,whereyouarewillingto tradeaccuracyforperformance.However,inthecaseofunsynchronizedreads,youaretradingawayagreaterdegreeofaccuracy,sincethevisible valueforasharedvariablecanbearbitrarilystale. MutableIntegerinListing3.2isnotthreadͲsafebecausethevaluefieldisaccessedfrombothgetandsetwithout synchronization.Amongotherhazards,itissusceptibletostalevalues:ifonethreadcallsset,otherthreadscallingget mayormaynotseethatupdate. WecanmakeMutableIntegerthreadsafebysynchronizingthegetterandsetterasshowninSynchronizedIntegerin Listing3.3.Synchronizingonlythesetterwouldnotbesufficient:threadscalling getwouldstillbeabletoseestale values. Listing3.2.NonǦthreadǦsafeMutableIntegerHolder. @NotThreadSafe public class MutableInteger { private int value; public int get() { return value; } public void set(int value) { this.value = value; } } Listing3.3.ThreadǦsafeMutableIntegerHolder. @ThreadSafe public class SynchronizedInteger { @GuardedBy("this") private int value; public synchronized int get() { return value; } public synchronized void set(int value) { this.value = value; } } 3.1.2.NonǦatomic64ǦbitOperations Whenathreadreadsavariablewithoutsynchronization,itmayseeastalevalue,butatleastitseesavaluethatwas actuallyplacedtherebysomethreadratherthansomerandomvalue.ThissafetyguaranteeiscalledoutͲofͲthinͲair safety. OutͲofͲthinͲairsafetyappliestoallvariables,withoneexception:64Ͳbitnumericvariables(doubleandlong)thatare notdeclaredvolatile(seeSection3.1.4).TheJavaMemoryModelrequiresfetchandstoreoperationstobeatomic, butfornonvolatilelonganddoublevariables,theJVMispermittedtotreata64Ͳbitreadorwriteastwoseparate32Ͳ bitoperations.Ifthereadsandwritesoccurindifferentthreads,itisthereforepossibletoreadanonvolatilelongand  25 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects getbackthehigh32bitsofonevalueandthelow32bitsofanother.[3]Thus,evenifyoudon'tcareaboutstalevalues,it is not safe to use shared mutable long and double variables in multithreaded programs unless they are declared volatileorguardedbyalock. [3]WhentheJavaVirtualMachineSpecificationwaswritten,manywidelyusedprocessorarchitecturescouldnotefficientlyprovideatomic64Ͳbit arithmeticoperations. 3.1.3.LockingandVisibility Intrinsic locking can be used to guarantee that one thread sees the effects of another in a predictable manner, as illustrated by Figure 3.1. When thread A executes a synchronized block, and subsequently thread B enters a synchronizedblockguardedbythesamelock,thevaluesofvariablesthatwerevisibletoApriortoreleasingthelock areguaranteedtobevisibletoBuponacquiringthelock.Inotherwords,everythingAdidinorpriortoasynchronized blockisvisibletoBwhenitexecutesasynchronizedblockguardedbythesamelock.Withoutsynchronization,thereis nosuchguarantee. Figure3.1.VisibilityGuaranteesforSynchronization.   Wecannowgivetheotherreasonfortherulerequiringallthreadstosynchronizeonthesamelockwhenaccessinga sharedmutablevariableͲtoguaranteethatvalueswrittenbyonethreadaremadevisibletootherthreads.Otherwise, ifathreadreadsavariablewithoutholdingtheappropriatelock,itmightseeastalevalue. Lockingisnotjustaboutmutualexclusion;itisalsoaboutmemoryvisibility.ToensurethatallthreadsseethemostupͲ toͲdatevaluesofsharedmutablevariables,thereadingandwritingthreadsmustsynchronizeonacommonlock. 3.1.4.VolatileVariables The Java language also provides an alternative, weaker form of synchronization, volatile variables, to ensure that updatestoavariablearepropagatedpredictablytootherthreads.Whenafieldisdeclaredvolatile,thecompilerand runtimeareputonnoticethatthisvariableissharedandthatoperationsonitshouldnotbereorderedwithother memory operations. Volatile variables are not cached in registers or in caches where they are hidden from other processors,soareadofavolatilevariablealwaysreturnsthemostrecentwritebyanythread. AgoodwaytothinkaboutvolatilevariablesistoimaginethattheybehaveroughlyliketheSynchronizedIntegerclass inListing3.3,replacingreadsandwritesofthevolatilevariablewithcallsto getand set.[4]Yetaccessingavolatile variableperformsnolockingandsocannotcausetheexecutingthreadtoblock,makingvolatilevariablesalighterͲ weightsynchronizationmechanismthansynchronized.[5]  26 JavaConcurrencyInPractice [4]Thisanalogyisnotexact;thememoryvisibilityeffectsofSynchronizedIntegerareactuallyslightlystrongerthanthoseofvolatilevariables.See Chapter16. [5]Volatilereadsareonlyslightlymoreexpensivethannonvolatilereadsonmostcurrentprocessorarchitectures. Thevisibilityeffectsofvolatilevariablesextendbeyondthevalueofthevolatilevariableitself.WhenthreadAwritesto avolatilevariableandsubsequentlythreadBreadsthatsamevariable,thevaluesofallvariablesthatwerevisibletoA priortowritingtothevolatilevariablebecomevisibletoBafterreadingthevolatilevariable.Sofromamemoryvisibility perspective,writingavolatilevariableislikeexitingasynchronizedblockandreadingavolatilevariableislikeentering asynchronizedblock.However,wedonotrecommendrelyingtooheavilyonvolatilevariablesforvisibility;codethat reliesonvolatilevariablesforvisibilityofarbitrarystateismorefragileandhardertounderstandthancodethatuses locking. Use volatilevariablesonlywhentheysimplifyimplementingandverifyingyoursynchronizationpolicy;avoidusing volatilevariableswhenverifyingcorrectnesswouldrequiresubtlereasoningaboutvisibility.Goodusesofvolatile variables include ensuring the visibility of their own state, that of the object they refer to, or indicating that an importantlifecycleevent(suchasinitializationorshutdown)hasoccurred. Listing3.4illustratesatypicaluseofvolatilevariables:checkingastatusflagtodeterminewhentoexitaloop.Inthis example,ouranthropomorphizedthreadistryingtogettosleepbythetimeͲhonoredmethodofcountingsheep.For thisexampletowork,theasleepflagmustbevolatile.Otherwise,thethreadmightnotnoticewhenasleephasbeen setbyanotherthread.[6]Wecouldinsteadhaveusedlockingtoensurevisibilityofchangestoasleep,butthatwould havemadethecodemorecumbersome. [6]Debuggingtip:Forserverapplications,besuretoalwaysspecifythe-serverJVMcommandlineswitchwheninvokingtheJVM,evenfor developmentandtesting.TheserverJVMperformsmoreoptimizationthantheclientJVM,suchashoistingvariablesoutofaloopthatarenot modifiedintheloop;codethatmightappeartoworkinthedevelopmentenvironment(clientJVM)canbreakinthedeploymentenvironment (serverJVM).Forexample,hadwe"forgotten"todeclarethevariableasleepasvolatileinListing3.4,theserverJVMcouldhoistthetestout oftheloop(turningitintoaninfiniteloop),buttheclientJVMwouldnot.Aninfiniteloopthatshowsupindevelopmentisfarlesscostlythanone thatonlyshowsupinproduction. Listing3.4.CountingSheep. volatile boolean asleep; ... while (!asleep) countSomeSheep(); Volatile variables are convenient, but they have limitations. The most common use for volatile variables is as a completion,interruption,orstatusflag,suchastheasleepflaginListing3.4.Volatilevariablescanbeusedforother kindsofstateinformation,butmorecareisrequiredwhenattemptingthis.Forexample,thesemanticsofvolatileare notstrongenoughtomaketheincrementoperation(count++)atomic,unlessyoucanguaranteethatthevariableis writtenonlyfromasinglethread.(AtomicvariablesdoprovideatomicreadͲmodifyͲwritesupportandcanoftenbeused as"bettervolatilevariables";seeChapter15.) Lockingcanguaranteebothvisibilityandatomicity;volatilevariablescanonlyguaranteevisibility. Youcanusevolatilevariablesonlywhenallthefollowingcriteriaaremet: x Writestothevariabledonotdependonitscurrentvalue,oryoucanensurethatonlyasinglethreadever updatesthevalue; x Thevariabledoesnotparticipateininvariantswithotherstatevariables;and x Lockingisnotrequiredforanyotherreasonwhilethevariableisbeingaccessed. 3.2.PublicationandEscape Publishinganobjectmeansmakingitavailabletocodeoutsideofitscurrentscope,suchasbystoringareferencetoit whereothercodecanfindit,returningitfromanonͲprivatemethod,orpassingittoamethodinanotherclass.Inmany situations,wewanttoensurethatobjectsandtheirinternalsarenotpublished.Inothersituations,wedowantto publishanobjectforgeneraluse,butdoingsoinathreadͲsafemannermayrequiresynchronization.Publishinginternal state variables can compromise encapsulation and make it more difficult to preserve invariants; publishing objects beforetheyarefullyconstructedcancompromisethreadsafety.Anobjectthatispublishedwhenitshouldnothave beenissaidtohaveescaped.Section3.5coversidiomsforsafepublication;rightnow,welookathowanobjectcan escape.  27 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects Themostblatantformofpublicationistostoreareferenceinapublicstaticfield,whereanyclassandthreadcouldsee it,asinListing3.5.TheinitializemethodinstantiatesanewHashSetandpublishesitbystoringareferencetoitinto knownSecrets. Listing3.5.PublishinganObject. public static Set knownSecrets; public void initialize() { knownSecrets = new HashSet(); } Publishingoneobjectmayindirectlypublishothers.IfyouaddaSecrettothepublishedknownSecretsset,you'vealso published that Secret, because any code can iterate the Set and obtain a reference to the new Secret. Similarly, returning a reference from a nonͲprivate method also publishes the returned object. UnsafeStates in Listing 3.6 publishesthesupposedlyprivatearrayofstateabbreviations. Listing3.6.AllowingInternalMutableStatetoEscape.Don'tDothis. class UnsafeStates { private String[] states = new String[] { "AK", "AL" ... }; public String[] getStates() { return states; } } Publishingstatesinthiswayisproblematicbecauseanycallercanmodifyitscontents.Inthiscase,thestatesarray hasescapeditsintendedscope,becausewhatwassupposedtobeprivatestatehasbeeneffectivelymadepublic. PublishinganobjectalsopublishesanyobjectsreferredtobyitsnonͲprivatefields.Moregenerally,anyobjectthatis reachablefromapublishedobjectbyfollowingsomechainofnonͲprivatefieldreferencesandmethodcallshasalso beenpublished. From the perspective of a class C, an alien method is one whose behavior is not fully specified by C. This includes methodsinotherclassesaswellasoverrideablemethods(neitherprivatenorfinal)inCitself.Passinganobjecttoan alienmethodmustalsobeconsideredpublishingthatobject.Sinceyoucan'tknowwhatcodewillactuallybeinvoked, youdon'tknowthatthealienmethodwon'tpublishtheobjectorretainareferencetoitthatmightlaterbeusedfrom anotherthread. Whetheranotherthreadactuallydoessomethingwithapublishedreferencedoesn'treallymatter,becausetheriskof misuseisstillpresent.[7]Onceanobjectescapes,youhavetoassumethatanotherclassorthreadmay,maliciouslyor carelessly,misuseit.Thisisacompellingreason touseencapsulation:it makesit practicaltoanalyzeprogramsfor correctnessandhardertoviolatedesignconstraintsaccidentally. [7]Ifsomeonestealsyourpasswordandpostsitonthealt.freeͲpasswordsnewsgroup,thatinformationhasescaped:whetherornotsomeonehas (yet)usedthosecredentialstocreatemischief,youraccounthasstillbeencompromised.Publishingareferenceposesthesamesortofrisk. Afinalmechanismbywhichanobjectoritsinternalstatecanbepublishedistopublishaninnerclassinstance,as shown in ThisEscape in Listing 3.7. When ThisEscape publishes the EventListener, it implicitly publishes the enclosing ThisEscape instance as well, because inner class instances contain a hidden reference to the enclosing instance.  28 JavaConcurrencyInPractice Listing3.7.ImplicitlyAllowingthethisReferencetoEscape.Don'tDothis. public class ThisEscape { public ThisEscape(EventSource source) { source.registerListener( new EventListener() { public void onEvent(Event e) { doSomething(e); } }); } } 3.2.1.SafeConstructionPractices ThisEscapeillustratesanimportantspecialcaseofescapeͲwhenthe thisreferencesescapesduringconstruction. Whentheinner EventListenerinstanceispublished,soistheenclosing ThisEscapeinstance.Butanobjectisina predictable,consistentstateonlyafteritsconstructorreturns,sopublishinganobjectfromwithinitsconstructorcan publishanincompletelyconstructedobject.Thisistrueevenifthepublicationisthelaststatementintheconstructor.If thethisreferenceescapesduringconstruction,theobjectisconsiderednotproperlyconstructed.[8] [8]Morespecifically,thethisreferenceshouldnotescapefromthethreaduntilaftertheconstructorreturns.Thethisreferencecanbestored somewherebytheconstructoraslongasitisnotusedbyanotherthreaduntilafterconstruction.SafeListenerinListing3.8usesthistechnique. Donotallowthethisreferencetoescapeduringconstruction. Acommonmistakethatcanletthethisreferenceescapeduringconstructionistostartathreadfromaconstructor. Whenanobjectcreatesathreadfromitsconstructor,italmostalwayssharesitsthisreferencewiththenewthread, eitherexplicitly(bypassingittotheconstructor)orimplicitly(becausetheThreadorRunnableisaninnerclassofthe owningobject). The new thread mightthenbeabletoseetheowningobjectbeforeitisfullyconstructed. There's nothingwrongwithcreatingathreadinaconstructor,butitisbestnottostartthethreadimmediately.Instead,expose a start or initialize method that starts the owned thread. (See Chapter 7 for more on service lifecycle issues.) Callinganoverrideableinstancemethod(onethatisneitherprivatenorfinal)fromtheconstructorcanalsoallowthe thisreferencetoescape. If you are tempted to register an event listener or start a thread from a constructor, you can avoid the improper constructionbyusingaprivateconstructorandapublicfactorymethod,asshowninSafeListenerinListing3.8. Listing3.8.UsingaFactoryMethodtoPreventthethisReferencefromEscapingDuringConstruction. public class SafeListener { private final EventListener listener; private SafeListener() { listener = new EventListener() { public void onEvent(Event e) { doSomething(e); } }; } public static SafeListener newInstance(EventSource source) { SafeListener safe = new SafeListener(); source.registerListener(safe.listener); return safe; } } 3.3.ThreadConfinement Accessingshared,mutabledatarequiresusingsynchronization;onewaytoavoidthisrequirementistonotshare.If dataisonlyaccessedfromasinglethread,nosynchronizationisneeded.Thistechnique,threadconfinement,isoneof thesimplestwaystoachievethreadsafety.Whenanobjectisconfinedtoathread,suchusageisautomaticallythreadͲ safeeveniftheconfinedobjectitselfisnot[CPJ2.3.2]. Swingusesthreadconfinementextensively.TheSwingvisualcomponentsanddatamodelobjectsarenotthreadsafe; instead,safetyisachievedbyconfiningthemtotheSwingeventdispatchthread.TouseSwingproperly,coderunningin threads other than the event thread should not access these objects. (To make this easier, Swing provides the  29 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects invokeLatermechanismtoscheduleaRunnableforexecutionintheeventthread.)ManyconcurrencyerrorsinSwing applicationsstemfromimproperuseoftheseconfinedobjectsfromanotherthread. AnothercommonapplicationofthreadconfinementistheuseofpooledJDBC(JavaDatabaseConnectivity)Connection objects.TheJDBCspecificationdoesnotrequirethatConnectionobjectsbethreadͲsafe.[9]Intypicalserverapplications, athreadacquiresaconnectionfromthepool,usesitforprocessingasinglerequest,andreturnsit.Sincemostrequests, suchasservletrequestsorEJB(EnterpriseJavaBeans)calls,areprocessedsynchronouslybyasinglethread,andthe poolwillnotdispensethesameconnectiontoanotherthreaduntilithasbeenreturned,thispatternofconnection managementimplicitlyconfinestheConnectiontothatthreadforthedurationoftherequest. [9]TheconnectionpoolimplementationsprovidedbyapplicationserversarethreadͲsafe;connectionpoolsarenecessarilyaccessedfrommultiple threads,soanonͲthreadͲsafeimplementationwouldnotmakesense. Justasthelanguagehasnomechanismforenforcingthatavariableisguardedbyalock,ithasnomeansofconfiningan object to a thread. Thread confinement is an element of your program's design that must be enforced by its implementation.ThelanguageandcorelibrariesprovidemechanismsthatcanhelpinmaintainingthreadconfinementͲ localvariablesandtheThreadLocalclassͲbutevenwiththese,itisstilltheprogrammer'sresponsibilitytoensurethat threadͲconfinedobjectsdonotescapefromtheirintendedthread. 3.3.1.AdǦhocThreadConfinement AdͲhocthreadconfinementdescribeswhentheresponsibilityformaintainingthreadconfinementfallsentirelyonthe implementation.AdͲhocthreadconfinementcanbefragilebecausenoneofthelanguagefeatures,suchasvisibility modifiersorlocalvariables,helpsconfinetheobjecttothetargetthread.Infact,referencestothreadͲconfinedobjects suchasvisualcomponentsordatamodelsinGUIapplicationsareoftenheldinpublicfields. Thedecisiontousethreadconfinementisoftenaconsequenceofthedecisiontoimplementaparticularsubsystem, suchastheGUI,asasingleͲthreadedsubsystem.SingleͲthreadedsubsystemscansometimesofferasimplicitybenefit thatoutweighsthefragilityofadͲhocthreadconfinement.[10] [10]AnotherreasontomakeasubsystemsingleͲthreadedisdeadlockavoidance;thisisoneoftheprimaryreasonsmostGUIframeworksare singleͲthreaded.SingleͲthreadedsubsystemsarecoveredinChapter9. Aspecialcaseofthreadconfinementappliestovolatilevariables.ItissafetoperformreadͲmodifyͲwriteoperationson sharedvolatilevariablesaslongasyouensurethatthevolatilevariableisonlywrittenfromasinglethread.Inthiscase, youareconfiningthemodificationtoasinglethreadtopreventraceconditions,andthevisibilityguaranteesforvolatile variablesensurethatotherthreadsseethemostupͲtoͲdatevalue. Becauseofitsfragility,adͲhocthreadconfinementshouldbeusedsparingly;ifpossible,useoneofthestrongerformsof threadconfinement(stackconfinementorThreadLocal)instead. 3.3.2.StackConfinement Stack confinement is a special case of thread confinement in which an object can only be reached through local variables.Justasencapsulationcanmakeiteasiertopreserveinvariants,localvariablescanmakeiteasiertoconfine objects to a thread. Local variables are intrinsically confined to the executing thread; they exist on the executing thread'sstack,whichisnotaccessibletootherthreads.Stackconfinement(alsocalledwithinͲthreadorthreadͲlocal usage,butnottobeconfusedwiththeThreadLocallibraryclass)issimplertomaintainandlessfragilethanadͲhoc threadconfinement. For primitively typed local variables, such as numPairs in loadTheArk in Listing 3.9, you cannot violate stack confinementevenifyoutried.Thereisnowaytoobtainareferencetoaprimitivevariable,sothelanguagesemantics ensurethatprimitivelocalvariablesarealwaysstackconfined.  30 JavaConcurrencyInPractice Listing3.9.ThreadConfinementofLocalPrimitiveandReferenceVariables. public int loadTheArk(Collection candidates) { SortedSet animals; int numPairs = 0; Animal candidate = null; // animals confined to method, don't let them escape! animals = new TreeSet(new SpeciesGenderComparator()); animals.addAll(candidates); for (Animal a : animals) { if (candidate == null || !candidate.isPotentialMate(a)) candidate = a; else { ark.load(new AnimalPair(candidate, a)); ++numPairs; candidate = null; } } return numPairs; } Maintainingstackconfinementforobjectreferencesrequiresalittlemoreassistancefromtheprogrammertoensure thatthereferentdoesnotescape.InloadTheArk,weinstantiateatreeSetandstoreareferencetoitinanimals.At thispoint,thereisexactlyonereferencetotheSet,heldinalocalvariableandthereforeconfinedtotheexecuting thread. However, if we were to publish a reference to the Set (or any of its internals), the confinement would be violatedandtheanimalswouldescape. Using a nonͲthreadͲsafe object in a withinͲthread context is still threadͲsafe. However, be careful: the design requirement that the object be confined to the executing thread, or the awareness that the confined object is not threadͲsafe,oftenexistsonlyintheheadofthedeveloperwhenthecodeiswritten.IftheassumptionofwithinͲthread usageisnotclearlydocumented,futuremaintainersmightmistakenlyallowtheobjecttoescape. 3.3.3.ThreadLocal AmoreformalmeansofmaintainingthreadconfinementisThreadLocal,whichallowsyoutoassociateaperͲthread valuewithavalueͲholdingobject.Thread-Localprovidesgetandsetaccessormethodsthatmaintainaseparatecopy of the value for each thread that uses it, soa get returns the most recent value passed to set from the currently executingthread. ThreadͲlocalvariablesareoftenusedtopreventsharingindesignsbasedonmutableSingletonsorglobalvariables.For example,asingleͲthreadedapplicationmightmaintainaglobaldatabaseconnectionthatisinitializedatstartuptoavoid having to pass a Connection to every method. Since JDBC connections may not be threadͲsafe, a multithreaded application that uses a global connection without additional coordination is not threadͲsafe either. By using a ThreadLocal to store the JDBC connection, as in ConnectionHolder in Listing 3.10, each thread will have its own connection. Listing3.10.UsingThreadLocaltoEnsurethreadConfinement. private static ThreadLocal connectionHolder = new ThreadLocal() { public Connection initialValue() { return DriverManager.getConnection(DB_URL); } }; public static Connection getConnection() { return connectionHolder.get(); } Thistechniquecanalsobeusedwhenafrequentlyusedoperationrequiresatemporaryobjectsuchasabufferand wantstoavoidreallocatingthetemporaryobjectoneachinvocation.Forexample,beforeJava5.0,Integer.toString usedaThreadLocaltostorethe12Ͳbytebufferusedforformattingitsresult,ratherthanusingasharedstaticbuffer (whichwouldrequirelocking)orallocatinganewbufferforeachinvocation.[11] [11]Thistechniqueisunlikelytobeaperformancewinunlesstheoperationisperformedveryfrequentlyortheallocationisunusuallyexpensive. InJava5.0,itwasreplacedwiththemorestraightforwardapproachofallocatinganewbufferforeveryinvocation,suggestingthatforsomething asmundaneasatemporarybuffer,itisnotaperformancewin. WhenathreadcallsThreadLocal.getforthefirsttime,initialValueisconsultedtoprovidetheinitialvalueforthat thread.Conceptually,youcanthinkofaThreadLocalasholdingaMapthatstoresthethreadͲspecific values,thoughthisisnothowitisactuallyimplemented.ThethreadͲspecificvaluesarestoredintheThreadobject itself;whenthethreadterminates,thethreadͲspecificvaluescanbegarbagecollected.  31 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects IfyouareportingasingleͲthreadedapplicationtoamultithreadedenvironment,youcanpreservethreadsafetyby convertingsharedglobalvariablesintoThreadLocals,ifthesemanticsofthesharedglobalspermitsthis;anapplicationͲ widecachewouldnotbeasusefulifitwereturnedintoanumberofthreadͲlocalcaches. ThreadLocal is widely used in implementing application frameworks. For example, J2EE containers associate a transactioncontextwithanexecutingthreadforthedurationofanEJBcall.Thisiseasilyimplementedusingastatic Thread-Localholdingthetransactioncontext:whenframeworkcodeneedstodeterminewhattransactioniscurrently running,itfetchesthetransactioncontextfromthisThreadLocal.Thisisconvenientinthatitreducestheneedtopass executioncontextinformationintoeverymethod,butcouplesanycodethatusesthismechanismtotheframework. ItiseasytoabuseThreadLocalbytreatingitsthreadconfinementpropertyasalicensetouseglobalvariablesorasa meansofcreating"hidden"methodarguments.Likeglobalvariables,threadͲlocalvariablescandetractfromreusability andintroducehiddencouplingsamongclasses,andshouldthereforebeusedwithcare. 3.4.Immutability TheotherendͲrunaroundtheneedtosynchronizeistouseimmutableobjects[EJItem13].Nearlyalltheatomicityand visibilityhazardswe'vedescribedsofar,suchasseeingstalevalues,losingupdates,orobservinganobjecttobeinan inconsistentstate,havetodowiththevagariesofmultiplethreadstryingtoaccessthesamemutablestateatthesame time.Ifanobject'sstatecannotbemodified,theserisksandcomplexitiessimplygoaway. An immutable object is one whose state cannot be changed after construction. Immutable objects are inherently threadͲsafe;theirinvariantsareestablishedbytheconstructor,andiftheirstatecannotbechanged,theseinvariants alwayshold. ImmutableobjectsarealwaysthreadͲsafe. Immutableobjectsaresimple.Theycanonlybeinonestate,whichiscarefullycontrolledbytheconstructor.Oneofthe mostdifficultelementsofprogramdesignisreasoningaboutthepossiblestatesofcomplexobjects.Reasoningabout thestateofimmutableobjects,ontheotherhand,istrivial. Immutable objects are also safer. Passing a mutable object to untrusted code, or otherwise publishing it where untrustedcodecouldfindit,isdangerousͲtheuntrustedcodemightmodifyitsstate,or,worse,retainareferencetoit and modify its state later from another thread. On the other hand, immutable objects cannot be subverted in this mannerbymaliciousorbuggycode,sotheyaresafetoshareandpublishfreelywithouttheneedtomakedefensive copies[EJItem24]. NeithertheJavaLanguageSpecificationnortheJavaMemoryModelformallydefinesimmutability,butimmutabilityis notequivalenttosimplydeclaringallfieldsofanobjectfinal.Anobjectwhosefieldsareallfinalmaystillbemutable, sincefinalfieldscanholdreferencestomutableobjects. Anobjectisimmutableif: x Itsstatecannotbemodifiedafterconstruction; x Allitsfieldsarefinal;[12]and x Itisproperlyconstructed(thethisreferencedoesnotescapeduringconstruction). [12]Itistechnicallypossibletohaveanimmutableobjectwithoutallfieldsbeingfinal.StringissuchaclassͲbutthisreliesondelicatereasoning aboutbenigndataracesthatrequiresadeepunderstandingoftheJavaMemoryModel.(Forthecurious:Stringlazilycomputesthehashcodethe firsttimehashCodeiscalledandcachesitinanonͲfinalfield,butthisworksonlybecausethatfieldcantakeononlyonenonͲdefaultvaluethatis thesameeverytimeitiscomputedbecauseitisderiveddeterministicallyfromimmutablestate.Don'ttrythisathome.) Immutable objects can still use mutable objects internally to manage their state, as illustrated by ThreeStooges in Listing3.11.WhiletheSetthatstoresthenamesismutable,thedesignofThreeStoogesmakesitimpossibletomodify thatSetafterconstruction.Thestoogesreferenceisfinal,soallobjectstateisreachedthroughafinalfield.Thelast requirement,properconstruction,iseasilymetsincetheconstructordoesnothingthatwouldcausethethisreference tobecomeaccessibletocodeotherthantheconstructoranditscaller.  32 JavaConcurrencyInPractice Listing3.11.ImmutableClassBuiltOutofMutableUnderlyingObjects. @Immutable public final class ThreeStooges { private final Set stooges = new HashSet(); public ThreeStooges() { stooges.add("Moe"); stooges.add("Larry"); stooges.add("Curly"); } public boolean isStooge(String name) { return stooges.contains(name); } } Becauseprogramstatechangesallthetime,youmightbetemptedtothinkthatimmutableobjectsareoflimiteduse, but this is not the case. There is a difference between an object being immutable and the reference to it being immutable.Programstatestoredinimmutableobjectscanstillbeupdatedby"replacing"immutableobjectswithanew instanceholdingnewstate;thenextsectionoffersanexampleofthistechnique.[13] [13]Manydevelopersfearthatthisapproachwillcreateperformanceproblems,butthesefearsareusuallyunwarranted.Allocationischeaper thanyoumightthink,andimmutableobjectsofferadditionalperformanceadvantagessuchasreducedneedforlockingordefensivecopiesand reducedimpactongenerationalgarbagecollection. 3.4.1.FinalFields Thefinalkeyword,amorelimitedversionoftheconstmechanismfromC++,supportstheconstructionofimmutable objects.Finalfieldscan'tbemodified(althoughtheobjectstheyrefertocanbemodifiediftheyaremutable),butthey alsohavespecialsemanticsundertheJavaMemoryModel.Itistheuseoffinalfieldsthatmakespossibletheguarantee of initialization safety (see Section 3.5.2) that lets immutable objects be freely accessed and shared without synchronization. Evenifanobjectismutable,makingsomefields finalcanstillsimplifyreasoningaboutitsstate,sincelimitingthe mutabilityofanobjectrestrictsitssetofpossible states.Anobject thatis"mostlyimmutable"buthasoneortwo mutablestatevariablesisstillsimplerthanonethathasmanymutablevariables.Declaringfieldsfinalalsodocuments tomaintainersthatthesefieldsarenotexpectedtochange. Justasitisagoodpracticetomakeallfieldsprivateunlesstheyneedgreatervisibility[EJItem12],itisagoodpractice tomakeallfieldsfinalunlesstheyneedtobemutable. 3.4.2.Example:UsingVolatiletoPublishImmutableObjects In UnsafeCachingFactorizeronpage24,wetriedtousetwo AtomicReferencestostorethelastnumberandlast factors,butthiswasnotthreadͲsafebecausewecouldnotfetchorupdatethetworelatedvaluesatomically.Using volatile variables for these values would not be threadͲsafe for the same reason. However, immutable objects can sometimesprovideaweakformofatomicity. The factoring servlet performs two operations that must be atomic: updating the cached result and conditionally fetchingthecachedfactorsifthecachednumbermatchestherequestednumber.Wheneveragroupofrelateddata itemsmustbeactedonatomically,considercreatinganimmutableholderclassforthem,suchasOneValueCache[14]in Listing3.12. [14]OneValueCachewouldn'tbeimmutablewithoutthecopyOfcallsintheconstructorandgetter.Arrays.copyOfwasaddedasaconveniencein Java6;clonewouldalsowork. Raceconditionsinaccessingorupdatingmultiplerelatedvariablescanbeeliminatedbyusinganimmutableobjectto hold all the variables. With a mutable holder object, you would have to use locking to ensure atomicity; with an immutableone,onceathreadacquiresareferencetoit,itneedneverworryaboutanotherthreadmodifyingitsstate.If thevariablesaretobeupdated,anewholderobjectiscreated,butanythreadsworkingwiththepreviousholderstill seeitinaconsistentstate.  33 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects Listing3.12.ImmutableHolderforCachingaNumberanditsFactors. @Immutable class OneValueCache { private final BigInteger lastNumber; private final BigInteger[] lastFactors; public OneValueCache(BigInteger i, BigInteger[] factors) { lastNumber = i; lastFactors = Arrays.copyOf(factors, factors.length); } public BigInteger[] getFactors(BigInteger i) { if (lastNumber == null || !lastNumber.equals(i)) return null; else return Arrays.copyOf(lastFactors, lastFactors.length); } } VolatileCachedFactorizerinListing3.13usesa OneValueCachetostorethecachednumberandfactors.Whena threadsetsthevolatile cachefieldtoreferenceanew OneValueCache,thenewcacheddatabecomesimmediately visibletootherthreads. ThecacheͲrelatedoperationscannotinterferewitheachotherbecauseOne-ValueCacheisimmutableandthecache field is accessed only once in each of the relevant code paths. This combination of an immutable holder object for multiple state variables related by an invariant, and a volatile reference used to ensure its timely visibility, allows VolatileCachedFactorizertobethreadͲsafeeventhoughitdoesnoexplicitlocking. 3.5.SafePublication Sofarwehavefocusedonensuringthatanobjectnotbepublished,suchaswhenitissupposedtobeconfinedtoa threadorwithinanotherobject.Ofcourse,sometimeswedowanttoshareobjectsacrossthreads,andinthiscasewe mustdososafely.Unfortunately,simplystoringareferencetoanobjectintoapublicfield,asinListing3.14,isnot enoughtopublishthatobjectsafely. Listing3.13.CachingtheLastResultUsingaVolatileReferencetoanImmutableHolderObject. @ThreadSafe public class VolatileCachedFactorizer implements Servlet { private volatile OneValueCache cache = new OneValueCache(null, null); public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req); BigInteger[] factors = cache.getFactors(i); if (factors == null) { factors = factor(i); cache = new OneValueCache(i, factors); } encodeIntoResponse(resp, factors); } } Listing3.14.PublishinganObjectwithoutAdequateSynchronization.Don'tDothis. // Unsafe publication public Holder holder; public void initialize() { holder = new Holder(42); } YoumaybesurprisedathowbadlythisharmlessͲlookingexamplecouldfail.Becauseofvisibilityproblems,theHolder couldappeartoanotherthreadtobeinaninconsistentstate,eventhoughitsinvariantswereproperlyestablishedbyits constructor!Thisimproperpublicationcouldallowanotherthreadtoobserveapartiallyconstructedobject. 3.5.1.ImproperPublication:WhenGoodObjectsGoBad You cannot rely on the integrity of partially constructed objects. An observing thread could see the object in an inconsistentstate,andthenlaterseeitsstatesuddenlychange,eventhoughithasnotbeenmodifiedsincepublication.  34 JavaConcurrencyInPractice Infact,iftheHolderinListing3.15ispublishedusingtheunsafepublicationidiominListing3.14,andathreadother thanthepublishingthreadweretocallassertSanity,itcouldthrowAssertionError![15] [15]TheproblemhereisnottheHolderclassitself,butthattheHolderisnotproperlypublished.However,Holdercanbemadeimmuneto improperpublicationbydeclaringthenfieldtobefinal,whichwouldmakeHolderimmutable;seeSection3.5.2. Listing3.15.ClassatRiskofFailureifNotProperlyPublished. public class Holder { private int n; public Holder(int n) { this.n = n; } public void assertSanity() { if (n != n) throw new AssertionError("This statement is false."); } } BecausesynchronizationwasnotusedtomaketheHoldervisibletootherthreads,wesaytheHolderwasnotproperly published.Twothingscangowrongwithimproperlypublishedobjects.Otherthreadscouldseeastalevalueforthe holderfield,andthusseeanullreferenceorotheroldervalueeventhoughavaluehasbeenplacedinholder.Butfar worse, other threads could see an up to date value for the holder reference, but stale values for the state of the Holder.[16]Tomakethingsevenlesspredictable,athreadmayseeastalevaluethefirsttimeitreadsafieldandthena moreupͲtoͲdatevaluethenexttime,whichiswhyassertSanitycanthrowAssertionError. [16]Whileitmayseemthatfieldvaluessetinaconstructorarethefirstvalueswrittentothosefieldsandthereforethatthereareno"older" valuestoseeasstalevalues,theObjectconstructorfirstwritesthedefaultvaluestoallfieldsbeforesubclassconstructorsrun.Itistherefore possibletoseethedefaultvalueforafieldasastalevalue. Attheriskofrepeatingourselves,someverystrangethingscanhappenwhendataissharedacrossthreadswithout sufficientsynchronization. 3.5.2.ImmutableObjectsandInitializationSafety Becauseimmutableobjectsaresoimportant,theJavaMemoryModeloffersaspecialguaranteeofinitializationsafety forsharingimmutableobjects.Aswe'veseen,thatanobjectreferencebecomesvisibletoanotherthreaddoesnot necessarilymeanthatthestateofthatobjectisvisibletotheconsumingthread.Inordertoguaranteeaconsistentview oftheobject'sstate,synchronizationisneeded. Immutableobjects,ontheotherhand,canbesafelyaccessedevenwhensynchronizationisnotusedtopublishthe objectreference.Forthisguaranteeofinitializationsafetytohold,alloftherequirementsforimmutabilitymustbemet: unmodifiable state, all fields are final, and proper construction. (If Holder in Listing 3.15 were immutable, assertSanitycouldnotthrowAssertionError,eveniftheHolderwasnotproperlypublished.) Immutableobjectscanbeusedsafelybyanythreadwithoutadditionalsynchronization,evenwhensynchronizationis notusedtopublishthem. Thisguaranteeextendstothevaluesofallfinalfieldsofproperlyconstructedobjects;finalfieldscanbesafelyaccessed withoutadditionalsynchronization.However,iffinalfieldsrefertomutableobjects,synchronizationisstillrequiredto accessthestateoftheobjectstheyreferto. 3.5.3.SafePublicationIdioms Objectsthatarenotimmutablemustbesafelypublished,whichusuallyentailssynchronizationbyboththepublishing andtheconsumingthread.Forthemoment,let'sfocusonensuringthattheconsumingthreadcanseetheobjectinits aspublishedstate;we'lldealwithvisibilityofmodificationsmadeafterpublicationsoon. To publish an object safely, both the reference to the object and the object's state must be made visible to other threadsatthesametime.Aproperlyconstructedobjectcanbesafelypublishedby: x Initializinganobjectreferencefromastaticinitializer; x StoringareferencetoitintoavolatilefieldorAtomicReference; x Storingareferencetoitintoafinalfieldofaproperlyconstructedobject;or x Storingareferencetoitintoafieldthatisproperlyguardedbyalock.  35 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ15BChapter3.Sharing Objects  TheinternalsynchronizationinthreadͲsafecollectionsmeansthatplacinganobjectinathreadͲsafecollection,suchasa Vector or synchronizedList, fulfills the last of these requirements. If thread A places object X in a threadͲsafe collectionandthreadBsubsequentlyretrievesit,BisguaranteedtoseethestateofXasAleftit,eventhoughthe applicationcodethathandsXoffinthismannerhasnoexplicitsynchronization.ThethreadͲsafelibrarycollectionsoffer thefollowingsafepublicationguarantees,eveniftheJavadocislessthanclearonthesubject: x PlacingakeyorvalueinaHashtable,synchronizedMap,orConcurrent-Mapsafelypublishesittoanythread thatretrievesitfromtheMap(whetherdirectlyorviaaniterator); x Placing an element in a Vector, CopyOnWriteArrayList, CopyOnWrite-ArraySet, synchronizedList, or synchronizedSetsafelypublishesittoanythreadthatretrievesitfromthecollection; x Placinganelementona BlockingQueueora ConcurrentLinkedQueuesafelypublishesittoanythreadthat retrievesitfromthequeue. Otherhandoffmechanismsintheclasslibrary(suchasFutureandExchanger)alsoconstitutesafepublication;wewill identifytheseasprovidingsafepublicationastheyareintroduced. Usingastaticinitializerisoftentheeasiestandsafestwaytopublishobjectsthatcanbestaticallyconstructed: public static Holder holder = new Holder(42); StaticinitializersareexecutedbytheJVMatclassinitializationtime;becauseofinternalsynchronizationintheJVM,this mechanismisguaranteedtosafelypublishanyobjectsinitializedinthisway[JLS12.4.2]. 3.5.4.EffectivelyImmutableObjects Safepublicationissufficientforotherthreadstosafelyaccessobjectsthatarenotgoingtobemodifiedafterpublication withoutadditionalsynchronization.ThesafepublicationmechanismsallguaranteethattheasͲpublishedstateofan objectisvisibletoallaccessingthreadsassoonasthereferencetoitisvisible,andifthatstateisnotgoingtobe changedagain,thisissufficienttoensurethatanyaccessissafe. Objectsthatarenottechnicallyimmutable,butwhosestatewillnotbemodifiedafterpublication,arecalledeffectively immutable.TheydonotneedtomeetthestrictdefinitionofimmutabilityinSection3.4;theymerelyneedtobetreated bytheprogramasiftheywereimmutableaftertheyarepublished.Usingeffectivelyimmutableobjectscansimplify developmentandimproveperformancebyreducingtheneedforsynchronization. Safelypublishedeffectivelyimmutableobjectscanbeusedsafelybyanythreadwithoutadditionalsynchronization. Forexample,Dateismutable,[17]butifyouuseitasifitwereimmutable,youmaybeabletoeliminatethelockingthat wouldotherwiseberequiredwhensharedaDateacrossthreads.SupposeyouwanttomaintainaMapstoringthelast logintimeofeachuser: [17]Thiswasprobablyamistakeintheclasslibrarydesign. public Map lastLogin = Collections.synchronizedMap(new HashMap()); IftheDatevaluesarenotmodifiedaftertheyareplacedintheMap,thenthesynchronizationinthesynchronizedMap implementation is sufficient to publish the Date values safely, and no additional synchronization is needed when accessingthem. 3.5.5.MutableObjects Ifanobjectmaybemodifiedafterconstruction,safepublicationensuresonlythevisibilityoftheasͲpublishedstate. Synchronizationmustbeusednotonlytopublishamutableobject,butalsoeverytimetheobjectisaccessedtoensure visibilityofsubsequentmodifications. Tosharemutableobjectssafely,theymustbesafelypublishedandbeeither threadͲsafeorguardedbyalock. Thepublicationrequirementsforanobjectdependonitsmutability: x Immutableobjectscanbepublishedthroughanymechanism; x Effectivelyimmutableobjectsmustbesafelypublished; x Mutableobjectsmustbesafelypublished,andmustbeeitherthreadͲsafeorguardedbyalock.  36 JavaConcurrencyInPractice 3.5.6.SharingObjectsSafely Wheneveryouacquireareferencetoanobject,youshouldknowwhatyouareallowedtodowithit.Doyouneedto acquirealockbeforeusingit?Areyouallowedtomodifyitsstate,oronlytoreadit?Manyconcurrencyerrorsstem fromfailingtounderstandthese"rulesofengagement"forasharedobject.Whenyoupublishanobject,youshould documenthowtheobjectcanbeaccessed. Themostusefulpoliciesforusingandsharingobjectsinaconcurrentprogramare: ThreadͲconfined.AthreadͲconfinedobjectisownedexclusivelybyandconfinedtoonethread,andcanbemodifiedby itsowningthread. Shared readͲonly. A shared readͲonly object can be accessed concurrently by multiple threads without additional synchronization,butcannotbe modifiedbyany thread.SharedreadͲonlyobjects includeimmutableandeffectively immutableobjects. SharedthreadͲsafe.AthreadͲsafeobjectperformssynchronizationinternally,somultiplethreadscanfreelyaccessit throughitspublicinterfacewithoutfurthersynchronization. Guarded. A guarded object can be accessed only with a specific lock held. Guarded objects include those that are encapsulatedwithinotherthreadͲsafeobjectsandpublishedobjectsthatareknowntobeguardedbyaspecificlock.   37 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects Chapter4.ComposingObjects Sofar,we'vecoveredthelowͲlevelbasicsofthreadsafetyandsynchronization.Butwedon'twanttohavetoanalyze eachmemoryaccesstoensurethatourprogramisthreadͲsafe;wewanttobeabletotakethreadͲsafecomponentsand safelycomposethemintolargercomponentsorprograms.Thischaptercoverspatternsforstructuringclassesthatcan make it easier to make them threadͲsafe and to maintain them without accidentally undermining their safety guarantees. 4.1.DesigningaThreadǦsafeClass WhileitispossibletowriteathreadͲsafeprogramthatstoresallitsstateinpublicstaticfields,itisalothardertoverify its thread safety or to modify it so that it remains threadͲsafe than one that uses encapsulation appropriately. EncapsulationmakesitpossibletodeterminethataclassisthreadͲsafewithouthavingtoexaminetheentireprogram. ThedesignprocessforathreadͲsafeclassshouldincludethesethreebasicelements: x Identifythevariablesthatformtheobject'sstate; x Identifytheinvariantsthatconstrainthestatevariables; x Establishapolicyformanagingconcurrentaccesstotheobject'sstate. Anobject'sstatestartswithitsfields.Iftheyareallofprimitivetype,thefieldscomprisetheentirestate.Counterin Listing4.1hasonlyonefield,sothevaluefieldcomprisesitsentirestate.Thestateofanobjectwithnprimitivefieldsis justthenͲtupleofitsfieldvalues;thestateofa2DPointisits(x,y)value.Iftheobjecthasfieldsthatarereferencesto other objects, its state will encompass fields from the referenced objects as well. For example, the state of a LinkedListincludesthestateofallthelinknodeobjectsbelongingtothelist. ThesynchronizationpolicydefineshowanobjectcoordinatesaccesstoitsstatewithoutviolatingitsinvariantsorpostͲ conditions.Itspecifieswhatcombinationofimmutability,threadconfinement,andlockingisusedtomaintainthread safety, and which variables are guarded by which locks. To ensure that the class can be analyzed and maintained, documentthesynchronizationpolicy. Listing4.1.SimpleThreadǦsafeCounterUsingtheJavaMonitorPattern. @ThreadSafe public final class Counter { @GuardedBy("this") private long value = 0; public synchronized long getValue() { return value; } public synchronized long increment() { if (value == Long.MAX_VALUE) throw new IllegalStateException("counter overflow"); return ++value; } } 4.1.1.GatheringSynchronizationRequirements Making a class threadͲsafe means ensuring that its invariants hold under concurrent access; this requires reasoning aboutitsstate.Objectsandvariableshaveastatespace:therangeofpossiblestatestheycantakeon.Thesmallerthis statespace,theeasieritistoreasonabout.Byusingfinalfieldswhereverpractical,youmakeitsimplertoanalyzethe possiblestatesanobjectcanbein.(Intheextremecase,immutableobjectscanonlybeinasinglestate.) Manyclasseshaveinvariantsthatidentifycertainstatesasvalidorinvalid.ThevaluefieldinCounterisalong.The state space of a long ranges from Long.MIN_VALUE to Long.MAX_VALUE, but Counter places constraints on value; negativevaluesarenotallowed. Similarly,operationsmayhavepostͲconditionsthatidentifycertainstatetransitionsasinvalid.Ifthecurrentstateofa Counteris17,theonlyvalidnextstateis18.Whenthenextstateisderivedfromthecurrentstate,theoperationis necessarilyacompoundaction.Notalloperationsimposestatetransitionconstraints;whenupdatingavariablethat holdsthecurrenttemperature,itspreviousstatedoesnotaffectthecomputation. ConstraintsplacedonstatesorstatetransitionsbyinvariantsandpostͲconditionscreateadditionalsynchronizationor encapsulation requirements. If certain states are invalid, then the underlying state variables must be encapsulated, otherwiseclientcodecouldputtheobjectintoaninvalidstate.Ifanoperationhasinvalidstatetransitions,itmustbe  38 JavaConcurrencyInPractice made atomic. On the other hand, if the class does not impose any such constraints, we may be able to relax encapsulationorserializationrequirementstoobtaingreaterflexibilityorbetterperformance. Aclasscanalsohaveinvariantsthatconstrainmultiplestatevariables.Anumberrangeclass,likeNumberRangeinListing 4.10,typicallymaintainsstatevariablesforthelowerandupperboundsoftherange.Thesevariablesmustobeythe constraintthatthelowerboundbelessthanorequaltotheupperbound.Multivariableinvariantslikethisonecreate atomicityrequirements:relatedvariablesmustbefetchedorupdatedinasingleatomicoperation.Youcannotupdate one,releaseandreacquirethelock,andthenupdatetheothers,sincethiscouldinvolveleavingtheobjectinaninvalid statewhenthelockwasreleased.Whenmultiplevariablesparticipateinaninvariant,thelockthatguardsthemmust beheldforthedurationofanyoperationthataccessestherelatedvariables. Youcannotensurethreadsafetywithoutunderstandinganobject'sinvariantsandpostͲconditions.Constraintsonthe validvaluesorstatetransitionsforstatevariablescancreateatomicityandencapsulationrequirements. 4.1.2.StateǦdependentOperations ClassinvariantsandmethodpostͲconditionsconstrainthevalidstatesandstatetransitionsforanobject.Someobjects alsohavemethodswithstateͲbasedpreconditions.Forexample,youcannotremoveanitemfromanemptyqueue;a queuemustbeinthe"nonempty"statebeforeyoucanremoveanelement.OperationswithstateͲbasedpreconditions arecalledstateͲdependent[CPJ3]. In a singleͲthreaded program, if a precondition does not hold, the operation has no choice but to fail. But in a concurrentprogram,thepreconditionmaybecometruelaterduetotheactionofanotherthread.Concurrentprograms addthepossibilityofwaitinguntilthepreconditionbecomestrue,andthenproceedingwiththeoperation. ThebuiltͲinmechanismsforefficientlywaitingforaconditiontobecometrueͲwaitandnotifyͲaretightlyboundto intrinsiclocking,andcanbedifficulttousecorrectly.Tocreateoperationsthatwaitforapreconditiontobecometrue beforeproceeding,itisofteneasiertouseexistinglibraryclasses,suchasblockingqueuesorsemaphores,toprovide the desired stateͲdependent behavior. Blocking library classes such as BlockingQueue, Semaphore, and other synchronizersarecoveredinChapter5;creatingstateͲdependentclassesusingthelowͲlevelmechanismsprovidedby theplatformandclasslibraryiscoveredinChapter14. 4.1.3.StateOwnership WeimpliedinSection4.1thatanobject'sstatecouldbeasubsetofthefieldsintheobjectgraphrootedatthatobject. Whymightitbeasubset?Underwhatconditionsarefieldsreachablefromagivenobjectnotpartofthatobject'sstate? Whendefiningwhichvariablesformanobject'sstate,wewanttoconsideronlythedatathatobjectowns.Ownershipis not embodied explicitly in the language, but is instead an element of class design. If you allocate and populate a HashMap, you are creating multiple objects: the HashMap object, a number of Map.Entry objects used by the implementationofHashMap,andperhapsotherinternalobjectsaswell.ThelogicalstateofaHashMapincludesthestate ofallitsMap.Entryandinternalobjects,eventhoughtheyareimplementedasseparateobjects. Forbetterorworse,garbagecollectionletsusavoidthinkingcarefullyaboutownership.Whenpassinganobjecttoa methodinC++,youhavetothinkfairlycarefullyaboutwhetheryouaretransferringownership,engaginginashortͲ termloan,orenvisioninglongͲtermjointownership.InJava,allthesesameownershipmodelsarepossible,butthe garbage collector reduces the cost of many of the common errors in reference sharing, enabling lessͲthanͲprecise thinkingaboutownership. Inmanycases,ownershipandencapsulationgotogetherͲtheobjectencapsulatesthestateitownsandownsthestate itencapsulates.Itistheownerofagivenstatevariablethatgetstodecideonthelockingprotocolusedtomaintainthe integrityofthatvariable'sstate.Ownershipimpliescontrol,butonceyoupublishareferencetoamutableobject,you nolongerhaveexclusivecontrol;atbest,youmighthave"sharedownership".Aclassusuallydoesnotowntheobjects passedtoitsmethodsorconstructors,unlessthemethodisdesignedtoexplicitlytransferownershipofobjectspassed in(suchasthesynchronizedcollectionwrapperfactorymethods). Collectionclassesoftenexhibitaformof"splitownership",inwhichthecollectionownsthestateofthecollection infrastructure,butclientcodeownstheobjectsstoredinthecollection.AnexampleisServletContextfromtheservlet framework. ServletContext provides a MapͲlike object container service to servlets where they can register and retrieveapplicationobjectsbynamewithsetAttributeandgetAttribute.TheServletContextobjectimplemented bytheservletcontainermustbethreadͲsafe,becauseitwillnecessarilybeaccessedbymultiplethreads.Servletsneed notusesynchronization whencalling set-Attributeand getAttribute,butthey mayhavetousesynchronization when using the objects stored in the ServletContext. These objects are owned by the application; they are being  39 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects storedforsafekeepingbytheservletcontainerontheapplication'sbehalf.Likeallsharedobjects,theymustbeshared safely; in order to prevent interference from multiple threads accessing the same object concurrently, they should eitherbethreadͲsafe,effectivelyimmutable,orexplicitlyguardedbyalock.[1] [1]Interestingly,theHttpSessionobject,whichperformsasimilarfunctionintheservletframework,mayhavestricterrequirements.Because theservletcontainermayaccesstheobjectsintheHttpSessionsotheycanbeserializedforreplicationorpassivation,theymustbethreadͲ safebecausethecontainerwillbeaccessingthemaswellasthewebapplication.(Wesay"mayhave"sincereplicationandpassivationisoutside oftheservletspecificationbutisacommonfeatureofservletcontainers.) 4.2.InstanceConfinement IfanobjectisnotthreadͲsafe,severaltechniquescanstillletitbeusedsafelyinamultithreadedprogram.Youcan ensurethatitisonlyaccessedfromasinglethread(threadconfinement),orthatallaccesstoitisproperlyguardedbya lock. EncapsulationsimplifiesmakingclassesthreadͲsafebypromotinginstanceconfinement,oftenjustcalledconfinement [CPJ2.3.3].Whenanobjectisencapsulatedwithinanotherobject,allcodepathsthathaveaccesstotheencapsulated object are known and can be therefore be analyzed more easily than if that object were accessible to the entire program. Combining confinement with an appropriate locking discipline can ensure that otherwise nonͲthreadͲsafe objectsareusedinathreadͲsafemanner. Encapsulatingdatawithinanobjectconfinesaccesstothedatatotheobject'smethods,makingiteasiertoensurethat thedataisalwaysaccessedwiththeappropriatelockheld. Confinedobjectsmustnotescapetheirintendedscope.Anobjectmaybeconfinedtoaclassinstance(suchasaprivate classmember),alexicalscope(suchasalocalvariable),orathread(suchasanobjectthatispassedfrommethodto methodwithinathread,butnotsupposedtobesharedacrossthreads).Objectsdon'tescapeontheirown,ofcourseͲ theyneedhelpfromthedeveloper,whoassistsbypublishingtheobjectbeyonditsintendedscope. PersonSetinListing4.2illustrateshowconfinementandlockingcanworktogethertomakeaclassthreadͲsafeeven whenitscomponentstatevariablesarenot.ThestateofPersonSetismanagedbyaHashSet,whichisnotthreadͲsafe. ButbecausemySetisprivateandnotallowedtoescape,theHashSetisconfinedtothePersonSet.Theonlycodepaths thatcanaccessmySetareaddPersonandcontainsPerson,andeachoftheseacquiresthelockonthePersonSet.Allits stateisguardedbyitsintrinsiclock,makingPersonSetthreadͲsafe. Listing4.2.UsingConfinementtoEnsureThreadSafety. @ThreadSafe public class PersonSet { @GuardedBy("this") private final Set mySet = new HashSet(); public synchronized void addPerson(Person p) { mySet.add(p); } public synchronized boolean containsPerson(Person p) { return mySet.contains(p); } } ThisexamplemakesnoassumptionsaboutthethreadͲsafetyofPerson,butifitismutable,additionalsynchronization willbeneededwhenaccessingaPersonretrievedfromaPersonSet.Themostreliablewaytodothiswouldbetomake PersonthreadͲsafe;lessreliablewouldbetoguardthePersonobjectswithalockandensurethatallclientsfollowthe protocolofacquiringtheappropriatelockbeforeaccessingthePerson. InstanceconfinementisoneoftheeasiestwaystobuildthreadͲsafeclasses.Italsoallowsflexibilityinthechoiceof lockingstrategy;PersonSethappenedtouseitsownintrinsiclocktoguarditsstate,butanylock,consistentlyused, woulddojustaswell.Instanceconfinementalsoallowsdifferentstatevariablestobeguardedbydifferentlocks.(Foran exampleofaclassthatusesmultiplelockobjectstoguarditsstate,seeServerStatuson236.) Therearemanyexamplesofconfinementintheplatformclasslibraries,includingsomeclassesthatexistsolelytoturn nonͲthreadͲsafe classes into threadͲsafe ones. The basic collection classes such as ArrayList and HashMap are not threadͲsafe,buttheclasslibraryprovideswrapperfactorymethods(Collections.synchronizedListandfriends)so theycanbeusedsafelyinmultithreadedenvironments.ThesefactoriesusetheDecoratorpattern(Gammaetal.,1995) towrapthecollectionwithasynchronizedwrapperobject;thewrapperimplementseachmethodoftheappropriate  40 JavaConcurrencyInPractice interface as a synchronized method that forwards the request to the underlying collection object. So long as the wrapperobjectholdstheonlyreachablereferencetotheunderlyingcollection(i.e.,theunderlyingcollectionisconfined tothewrapper),thewrapperobjectisthenthreadͲsafe.TheJavadocforthesemethodswarnsthatallaccesstothe underlyingcollectionmustbemadethroughthewrapper. Ofcourse,itisstillpossibletoviolateconfinementbypublishingasupposedlyconfinedobject;ifanobjectisintended tobeconfinedtoaspecificscope,thenlettingitescapefromthatscopeisabug.Confinedobjectscanalsoescapeby publishingotherobjectssuchasiteratorsorinnerclassinstancesthatmayindirectlypublishtheconfinedobjects. ConfinementmakesiteasiertobuildthreadͲsafeclassesbecauseaclassthatconfinesitsstatecanbeanalyzedfor threadsafetywithouthavingtoexaminethewholeprogram. 4.2.1.TheJavaMonitorPattern FollowingtheprincipleofinstanceconfinementtoitslogicalconclusionleadsyoutotheJavamonitorpattern.[2]An objectfollowingtheJavamonitorpatternencapsulatesallitsmutablestateandguardsitwiththeobject'sownintrinsic lock. [2]TheJavamonitorpatternisinspiredbyHoare'sworkonmonitors(Hoare,1974),thoughtherearesignificantdifferencesbetweenthispattern andatruemonitor.Thebytecodeinstructionsforenteringandexitingasynchronizedblockareevencalledmonitorenterandmonitorexit,and Java'sbuiltͲin(intrinsic)locksaresometimescalledmonitorlocksormonitors. CounterinListing4.1showsatypicalexampleofthispattern.Itencapsulatesonestatevariable,value,andallaccess tothatstatevariableisthroughthemethodsofCounter,whichareallsynchronized. The Java monitor pattern is used by many library classes, such as Vector and Hashtable. Sometimes a more sophisticated synchronization policy is needed; Chapter 11 shows how to improve scalability through finerͲgrained lockingstrategies.TheprimaryadvantageoftheJavamonitorpatternisitssimplicity. TheJavamonitorpatternismerelyaconvention;anylockobjectcouldbeusedtoguardanobject'sstatesolongasitis usedconsistently.Listing4.3illustratesaclassthatusesaprivatelocktoguarditsstate. Listing4.3.GuardingStatewithaPrivateLock. public class PrivateLock { private final Object myLock = new Object(); @GuardedBy("myLock") Widget widget; void someMethod() { synchronized(myLock) { // Access or modify the state of widget } } } Thereareadvantagestousingaprivatelockobjectinsteadofanobject'sintrinsiclock(oranyotherpubliclyaccessible lock).Makingthelockobjectprivateencapsulatesthelocksothatclientcodecannotacquireit,whereasapublicly accessible lock allows client code to participate in its synchronization policyͲcorrectly or incorrectly. Clients that improperlyacquireanotherobject'slockcouldcauselivenessproblems,andverifyingthatapubliclyaccessiblelockis properlyusedrequiresexaminingtheentireprogramratherthanasingleclass. 4.2.2.Example:TrackingFleetVehicles Counter in Listing 4.1 is a concise, but trivial, example of the Java monitor pattern. Let's build a slightly less trivial example:a"vehicletracker"fordispatchingfleetvehiclessuchastaxicabs,policecars,ordeliverytrucks.We'llbuildit first using the monitor pattern, and then see how to relax some of the encapsulation requirements while retaining threadsafety. EachvehicleisidentifiedbyaStringandhasalocationrepresentedby(x,y)coordinates.TheVehicleTrackerclasses encapsulatetheidentityandlocationsoftheknownvehicles,makingthemwellͲsuitedasadatamodelinamodelviewͲ controllerGUIapplicationwhereitmightbesharedbyaviewthreadandmultipleupdaterthreads.Theviewthread wouldfetchthenamesandlocationsofthevehiclesandrenderthemonadisplay: Map locations = vehicles.getLocations(); for (String key : locations.keySet()) renderVehicle(key, locations.get(key));   41 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects Similarly,theupdaterthreadswouldmodifyvehiclelocationswithdatareceivedfromGPSdevicesorenteredmanually byadispatcherthroughaGUIinterface: void vehicleMoved(VehicleMovedEvent evt) { Point loc = evt.getNewLocation(); vehicles.setLocation(evt.getVehicleId(), loc.x, loc.y); } Sincetheviewthreadandtheupdaterthreadswillaccessthedatamodelconcurrently,itmustbethreadͲsafe.Listing 4.4showsanimplementationofthevehicletrackerusingtheJavamonitorpatternthatusesMutablePointinListing4.5 forrepresentingthevehiclelocations. EventhoughMutablePointisnotthreadͲsafe,thetrackerclassis.Neitherthemapnoranyofthemutablepointsit containsiseverpublished.Whenweneedtoareturnvehiclelocationstocallers,theappropriatevaluesarecopied usingeithertheMutablePointcopyconstructorordeepCopy,whichcreatesanewMapwhosevaluesarecopiesofthe keysandvaluesfromtheoldMap.[3] [3]NotethatdeepCopycan'tjustwraptheMapwithanunmodifiableMap,becausethatprotectsonlythecollectionfrommodification;itdoesnot preventcallersfrommodifyingthemutableobjectsstoredinit.Forthesamereason,populatingtheHashMapindeepCopyviaacopyconstructor wouldn'tworkeither,becauseonlythereferencestothepointswouldbecopied,notthepointobjectsthemselves. Thisimplementationmaintainsthreadsafetyinpartbycopyingmutabledatabeforereturningittotheclient.Thisis usuallynotaperformanceissue,butcouldbecomeoneifthesetofvehiclesisverylarge.[4]Anotherconsequenceof copyingthedataoneachcalltogetLocationisthatthecontentsofthereturnedcollectiondonotchangeevenifthe underlyinglocationschange.Whetherthisisgoodorbaddependsonyourrequirements.Itcouldbeabenefitifthere areinternalconsistencyrequirementsonthelocationset,inwhichcasereturningaconsistentsnapshotiscritical,ora drawbackifcallersrequireupͲtoͲdateinformationforeachvehicleandthereforeneedtorefreshtheirsnapshotmore often. [4]BecausedeepCopyiscalledfromasynchronizedmethod,thetracker'sintrinsiclockisheldforthedurationofwhatmightbealongͲrunning copyoperation,andthiscoulddegradetheresponsivenessoftheuserinterfacewhenmanyvehiclesarebeingtracked. 4.3.DelegatingThreadSafety Allbutthemosttrivialobjectsarecompositeobjects.TheJavamonitorpatternisusefulwhenbuildingclassesfrom scratchorcomposingclassesoutofobjectsthatarenotthreadͲsafe.Butwhatifthecomponentsofourclassarealready threadͲsafe?Doweneedtoaddanadditionallayerofthreadsafety?Theansweris..."itdepends".Insomecasesa compositemadeofthreadͲsafecomponentsisthreadͲsafe(Listings4.7and4.9),andinothersitismerelyagoodstart (4.10). In CountingFactorizer on page 23, we added an AtomicLong to an otherwise stateless object, and the resulting composite object was still threadͲsafe. Since the state of CountingFactorizer is the state of the threadͲsafe AtomicLong,andsinceCountingFactorizerimposesnoadditionalvalidityconstraintsonthestateofthecounter,itis easy to see that CountingFactorizer is threadͲsafe. We could say that CountingFactorizer delegates its thread safetyresponsibilitiestotheAtomicLong:CountingFactorizeristhreadͲsafebecauseAtomicLongis.[5] [5]Ifcountwerenotfinal,thethreadsafetyanalysisofCountingFactorizerwouldbemorecomplicated.IfCountingFactorizercouldmodifycount toreferenceadifferentAtomicLong,wewouldthenhavetoensurethatthisupdatewasvisibletoallthreadsthatmightaccessthecount,andthat therewerenoraceconditionsregardingthevalueofthecountreference.Thisisanothergoodreasontousefinalfieldswhereverpractical.  42 JavaConcurrencyInPractice Listing4.4.MonitorǦbasedVehicleTrackerImplementation. @ThreadSafe public class MonitorVehicleTracker { @GuardedBy("this") private final Map locations; public MonitorVehicleTracker( Map locations) { this.locations = deepCopy(locations); } public synchronized Map getLocations() { return deepCopy(locations); } public synchronized MutablePoint getLocation(String id) { MutablePoint loc = locations.get(id); return loc == null ? null : new MutablePoint(loc); } public synchronized void setLocation(String id, int x, int y) { MutablePoint loc = locations.get(id); if (loc == null) throw new IllegalArgumentException("No such ID: " + id); loc.x = x; loc.y = y; } private static Map deepCopy( Map m) { Map result = new HashMap(); for (String id : m.keySet()) result.put(id, new MutablePoint(m.get(id))); return Collections.unmodifiableMap(result); } } public class MutablePoint { /* Listing 4.5 */ } Listing4.5.MutablePointClassSimilartoJava.awt.Point. @NotThreadSafe public class MutablePoint { public int x, y; public MutablePoint() { x = 0; y = 0; } public MutablePoint(MutablePoint p) { this.x = p.x; this.y = p.y; } } 4.3.1.Example:VehicleTrackerUsingDelegation Asamoresubstantialexampleofdelegation,let'sconstructaversionofthevehicletrackerthatdelegatestoathreadͲ safeclass.WestorethelocationsinaMap,sowestartwithathreadͲsafeMapimplementation,ConcurrentHashMap.We alsostorethelocationusinganimmutablePointclassinsteadofMutablePoint,showninListing4.6. Listing4.6.ImmutablePointclassusedbyDelegatingVehicleTracker. @Immutable public class Point { public final int x, y; public Point(int x, int y) { this.x = x; this.y = y; } } PointisthreadͲsafebecauseitisimmutable.Immutablevaluescanbefreelysharedandpublished,sowenolonger needtocopythelocationswhenreturningthem. DelegatingVehicleTrackerinListing4.7doesnotuseanyexplicitsynchronization;allaccesstostateismanagedby ConcurrentHashMap,andallthekeysandvaluesoftheMapareimmutable.  43 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects If we had used the original MutablePoint class instead of Point, we would be breaking encapsulation by letting getLocationspublishareferencetomutablestatethatisnotthreadͲsafe.Noticethatwe'vechangedthebehaviorof thevehicletrackerclassslightly;whilethemonitorversionreturnedasnapshotofthelocations,thedelegatingversion returnsanunmodifiablebut"live"viewofthevehiclelocations.ThismeansthatifthreadAcallsgetLocationsand threadBlatermodifiesthelocationofsomeofthepoints,thosechangesarereflectedintheMapreturnedtothreadA. Asweremarkedearlier,thiscanbeabenefit(moreupͲtoͲdatedata)oraliability(potentiallyinconsistentviewofthe fleet),dependingonyourrequirements. Ifanunchangingviewofthefleetisrequired,getLocationscouldinsteadreturnashallowcopyofthelocationsmap. SincethecontentsoftheMapareimmutable,onlythestructureoftheMap,notthecontents,mustbecopied,asshown inListing4.8(whichreturnsaplainHashMap,sincegetLocationsdidnotpromisetoreturnathreadͲsafeMap). Listing4.7.DelegatingThreadSafetytoaConcurrentHashMap. @ThreadSafe public class DelegatingVehicleTracker { private final ConcurrentMap locations; private final Map unmodifiableMap; public DelegatingVehicleTracker(Map points) { locations = new ConcurrentHashMap(points); unmodifiableMap = Collections.unmodifiableMap(locations); } public Map getLocations() { return unmodifiableMap; } public Point getLocation(String id) { return locations.get(id); } public void setLocation(String id, int x, int y) { if (locations.replace(id, new Point(x, y)) == null) throw new IllegalArgumentException( "invalid vehicle name: " + id); } } Listing4.8.ReturningaStaticCopyoftheLocationSetInsteadofa"Live"One. public Map getLocations() { return Collections.unmodifiableMap( new HashMap(locations)); } 4.3.2.IndependentStateVariables Thedelegationexamplessofardelegatetoasingle,threadͲsafestatevariable.Wecanalsodelegatethreadsafetyto morethanoneunderlyingstatevariableaslongasthoseunderlyingstatevariablesareindependent,meaningthatthe compositeclassdoesnotimposeanyinvariantsinvolvingthemultiplestatevariables. VisualComponent in Listing 4.9 is a graphical component that allows clients to register listeners for mouse and keystrokeevents.Itmaintainsalistofregisteredlistenersofeachtype,sothatwhenaneventoccurstheappropriate listenerscanbeinvoked.Butthereisnorelationshipbetweenthesetofmouselistenersandkeylisteners;thetwoare independent,andthereforeVisualComponentcandelegateitsthreadsafetyobligationstotwounderlyingthreadͲsafe lists. VisualComponentusesaCopyOnWriteArrayListtostoreeachlistenerlist;thisisathreadͲsafeListimplementation particularlysuitedformanaginglistenerlists(seeSection5.2.3).Each LististhreadͲsafe,andbecausethereareno constraints coupling the state of one to the state of the other, VisualComponent can delegate its thread safety responsibilitiestotheunderlyingmouseListenersandkeyListenersobjects.  44 JavaConcurrencyInPractice Listing4.9.DelegatingThreadSafetytoMultipleUnderlyingStateVariables. public class VisualComponent { private final List keyListeners = new CopyOnWriteArrayList(); private final List mouseListeners = new CopyOnWriteArrayList(); public void addKeyListener(KeyListener listener) { keyListeners.add(listener); } public void addMouseListener(MouseListener listener) { mouseListeners.add(listener); } public void removeKeyListener(KeyListener listener) { keyListeners.remove(listener); } public void removeMouseListener(MouseListener listener) { mouseListeners.remove(listener); } } 4.3.3.WhenDelegationFails MostcompositeclassesarenotassimpleasVisualComponent:theyhaveinvariantsthatrelatetheircomponentstate variables. NumberRange in Listing 4.10 uses two AtomicIntegers to manage its state, but imposes an additional constraintͲthatthefirstnumberbelessthanorequaltothesecond. NumberRangeisnotthreadͲsafe;itdoesnotpreservetheinvariantthatconstrainslowerandupper.ThesetLowerand setUppermethodsattempttorespectthisinvariant,butdosopoorly.BothsetLowerandsetUpperarecheckͲthenͲact sequences,buttheydonotusesufficientlockingtomakethematomic.Ifthenumberrangeholds(0,10),andone threadcallssetLower(5)whileanotherthreadcallssetUpper(4),withsomeunluckytimingbothwillpassthechecks inthesettersandbothmodificationswillbeapplied.Theresultisthattherangenowholds(5,4)aninvalidstate.So whiletheunderlyingAtomicIntegersarethreadͲsafe,thecompositeclassisnot.Becausetheunderlyingstatevariables lower and upper are not independent, NumberRange cannot simply delegate thread safety to its threadͲsafe state variables. NumberRangecouldbemadethreadͲsafebyusinglockingtomaintainitsinvariants,suchasguardinglowerandupper withacommonlock.Itmustalsoavoidpublishingloweranduppertopreventclientsfromsubvertingitsinvariants. Ifaclasshascompoundactions,as NumberRangedoes,delegationaloneisagainnotasuitableapproachforthread safety.Inthesecases,theclassmustprovideitsownlockingtoensurethatcompoundactionsareatomic,unlessthe entirecompoundactioncanalsobedelegatedtotheunderlyingstatevariables. IfaclassiscomposedofmultipleindependentthreadͲsafestatevariablesandhasnooperationsthathaveanyinvalid statetransitions,thenitcandelegatethreadsafetytotheunderlyingstatevariables.   45 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects Listing4.10.NumberRangeClassthatdoesNotSufficientlyProtectItsInvariants.Don'tDothis. public class NumberRange { // INVARIANT: lower <= upper private final AtomicInteger lower = new AtomicInteger(0); private final AtomicInteger upper = new AtomicInteger(0); public void setLower(int i) { // Warning -- unsafe check-then-act if (i > upper.get()) throw new IllegalArgumentException( "can't set lower to " + i + " > upper"); lower.set(i); } public void setUpper(int i) { // Warning -- unsafe check-then-act if (i < lower.get()) throw new IllegalArgumentException( "can't set upper to " + i + " < lower"); upper.set(i); } public boolean isInRange(int i) { return (i >= lower.get() && i <= upper.get()); } } TheproblemthatpreventedNumberRangefrombeingthreadͲsafeeventhoughitsstatecomponentswerethreadͲsafeis very similar to one of the rules about volatile variables described in Section 3.1.4: a variable is suitable for being declaredvolatileonlyifitdoesnotparticipateininvariantsinvolvingotherstatevariables. 4.3.4.PublishingUnderlyingStateVariables Whenyoudelegatethreadsafetytoanobject'sunderlyingstatevariables,underwhatconditionscanyoupublishthose variables so that other classes can modify them as well? Again, the answer depends on what invariants your class imposes on those variables. While the underlying value field in Counter could take on any integer value, Counter constrainsittotakeononlypositivevalues,andtheincrementoperationconstrainsthesetofvalidnextstatesgiven anycurrentstate.Ifyouweretomakethevaluefieldpublic,clientscouldchangeittoaninvalidvalue,sopublishingit wouldrendertheclassincorrect.Ontheotherhand,ifavariablerepresentsthecurrenttemperatureortheIDofthe lastusertologon,thenhavinganotherclassmodifythisvalueatanytimeprobablywouldnotviolateanyinvariants,so publishing this variable might be acceptable. (It still may not be a good idea, since publishing mutable variables constrains future development and opportunities for subclassing, but it would not necessarily render the class not threadͲsafe.) IfastatevariableisthreadͲsafe,doesnotparticipateinanyinvariantsthatconstrainitsvalue,andhasnoprohibited statetransitionsforanyofitsoperations,thenitcansafelybepublished. For example, it would be safe to publish mouseListeners or keyListeners in VisualComponent. Because VisualComponentdoesnotimposeanyconstraintsonthevalidstatesofitslistenerlists,thesefieldscouldbemade publicorotherwisepublishedwithoutcompromisingthreadsafety. 4.3.5.Example:VehicleTrackerthatPublishesItsState Let'sconstructanotherversionofthevehicletrackerthatpublishesitsunderlyingmutablestate.Again,weneedto modifytheinterfacealittlebittoaccommodatethischange,thistimeusingmutablebutthreadͲsafepoints.  46 JavaConcurrencyInPractice Listing4.11.ThreadǦsafeMutablePointClass. @ThreadSafe public class SafePoint { @GuardedBy("this") private int x, y; private SafePoint(int[] a) { this(a[0], a[1]); } public SafePoint(SafePoint p) { this(p.get()); } public SafePoint(int x, int y) { this.x = x; this.y = y; } public synchronized int[] get() { return new int[] { x, y }; } public synchronized void set(int x, int y) { this.x = x; this.y = y; } } SafePointinListing4.11providesagetterthatretrievesboththexandyvaluesatoncebyreturningatwoͲelement array.[6]Ifweprovidedseparategettersforxandy,thenthevaluescouldchangebetweenthetimeonecoordinateis retrievedandtheother,resultinginacallerseeinganinconsistentvalue:an(x,y)locationwherethevehicleneverwas. UsingSafePoint,wecanconstructavehicletrackerthatpublishestheunderlyingmutablestatewithoutundermining threadsafety,asshowninthePublishingVehicleTrackerclassinListing4.12. [6]Theprivateconstructorexiststoavoidtheraceconditionthatwouldoccurifthecopyconstructorwereimplementedasthis(p.x,p.y);thisisan exampleoftheprivateconstructorcaptureidiom(BlochandGafter,2005). PublishingVehicleTrackerderivesitsthreadsafetyfromdelegationtoanunderlyingConcurrentHashMap,butthis timethecontentsoftheMaparethreadͲsafemutablepointsratherthanimmutableones.ThegetLocationmethod returns an unmodifiable copy of the underlying Map. Callers cannot add or remove vehicles, but could change the locationofoneofthevehiclesbymutatingtheSafePointvaluesinthereturnedMap.Again,the"live"natureoftheMap maybeabenefitoradrawback,dependingontherequirements.PublishingVehicleTrackeristhreadͲsafe,butwould notbesoifitimposedanyadditionalconstraintsonthevalidvaluesforvehiclelocations.Ifitneededtobeableto "veto" changes to vehicle locations or to take action when a location changes, the approach taken by PublishingVehicleTrackerwouldnotbeappropriate. Listing4.12.VehicleTrackerthatSafelyPublishesUnderlyingState. @ThreadSafe public class PublishingVehicleTracker { private final Map locations; private final Map unmodifiableMap; public PublishingVehicleTracker( Map locations) { this.locations = new ConcurrentHashMap(locations); this.unmodifiableMap = Collections.unmodifiableMap(this.locations); } public Map getLocations() { return unmodifiableMap; } public SafePoint getLocation(String id) { return locations.get(id); } public void setLocation(String id, int x, int y) { if (!locations.containsKey(id)) throw new IllegalArgumentException( "invalid vehicle name: " + id); locations.get(id).set(x, y); } }   47 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects 4.4.AddingFunctionalitytoExistingThreadǦsafeClasses The Java class library contains many useful "building block" classes. Reusing existing classes is often preferable to creating new ones: reuse can reduce development effort, development risk (because the existing components are already tested), and maintenance cost. Sometimes a threadͲsafe class that supports all of the operations we want alreadyexists,butoftenthebestwecanfindisaclassthatsupportsalmostalltheoperationswewant,andthenwe needtoaddanewoperationtoitwithoutunderminingitsthreadsafety. Asanexample,let'ssayweneedathreadͲsafeListwithanatomicputͲifͲabsentoperation.ThesynchronizedList implementationsnearlydothejob,sincetheyprovidethecontainsandaddmethodsfromwhichwecanconstructa putͲifͲabsentoperation. TheconceptofputͲifͲabsentisstraightforwardenoughͲchecktoseeifanelementisinthecollectionbeforeaddingit, anddonotadditifitisalreadythere.(Your"checkͲthenͲact"warningbellsshouldbegoingoffnow.)Therequirement thattheclassbethreadͲsafeimplicitlyaddsanotherrequirementͲthatoperationslikeputͲifͲabsentbeatomic.Any reasonableinterpretationsuggeststhat,ifyoutakeaListthatdoesnotcontainobjectX,andaddXtwicewithputͲifͲ absent,theresultingcollectioncontainsonlyonecopyofX.But,ifputͲifͲabsentwerenotatomic,withsomeunlucky timingtwothreadscouldbothseethatXwasnotpresentandbothaddX,resultingintwocopiesofX. Thesafestwaytoaddanewatomicoperationistomodifytheoriginalclasstosupportthedesiredoperation,butthisis notalwayspossiblebecauseyoumaynothaveaccesstothesourcecodeormaynotbefreetomodifyit.Ifyoucan modifytheoriginalclass,youneedtounderstandtheimplementation'ssynchronizationpolicysothatyoucanenhance itinamannerconsistentwithitsoriginaldesign.Addingthenewmethoddirectlytotheclassmeansthatallthecode that implements the synchronization policy for that class is still contained in one source file, facilitating easier comprehensionandmaintenance. Anotherapproachistoextendtheclass,assumingitwasdesignedforextension.BetterVectorinListing4.13extends VectortoaddaputIfAbsentmethod.ExtendingVectorisstraightforwardenough,butnotallclassesexposeenough oftheirstatetosubclassestoadmitthisapproach. Extensionismorefragilethanaddingcodedirectlytoaclass,becausetheimplementationofthesynchronizationpolicy is now distributed over multiple, separately maintained source files. If the underlying class were to change its synchronizationpolicybychoosingadifferentlocktoguarditsstatevariables,thesubclasswouldsubtlyandsilently break, because it no longer used the right lock to control concurrent access to the base class's state. (The synchronizationpolicyofVectorisfixedbyitsspecification,soBetterVectorwouldnotsufferfromthisproblem.) Listing4.13.ExtendingVectortohaveaPutǦifǦabsentMethod. @ThreadSafe public class BetterVector extends Vector { public synchronized boolean putIfAbsent(E x) { boolean absent = !contains(x); if (absent) add(x); return absent; } } 4.4.1.ClientǦsideLocking ForanArrayListwrappedwithaCollections.synchronizedListwrapper,neitheroftheseapproachesͲaddinga methodtotheoriginalclassorextendingtheclassͲworksbecausetheclientcodedoesnotevenknowtheclassofthe Listobjectreturnedfromthesynchronizedwrapperfactories.Athirdstrategyistoextendthefunctionalityoftheclass withoutextendingtheclassitselfbyplacingextensioncodeina"helper"class. Listing4.14showsafailedattempttocreateahelperclasswithanatomicputͲifͲabsentoperationforoperatingona threadͲsafeList.  48 JavaConcurrencyInPractice Listing4.14.NonǦthreadǦsafeAttempttoImplementPutǦifǦabsent.Don'tDothis. @NotThreadSafe public class ListHelper { public List list = Collections.synchronizedList(new ArrayList()); ... public synchronized boolean putIfAbsent(E x) { boolean absent = !list.contains(x); if (absent) list.add(x); return absent; } } Whywouldn'tthiswork?Afterall, putIfAbsentis synchronized,right?Theproblemisthatitsynchronizesonthe wrong lock. Whatever lock the List uses to guard its state, it sure isn't the lock on the ListHelper. ListHelper providesonlytheillusionofsynchronization;thevariouslistoperations,whileallsynchronized,usedifferentlocks, whichmeansthatputIfAbsentisnotatomicrelativetootheroperationsontheList.Sothereisnoguaranteethat anotherthreadwon'tmodifythelistwhileputIfAbsentisexecuting. Tomakethisapproachwork,wehavetousethesamelockthattheListusesbyusingclientͲsidelockingorexternal locking.ClientͲsidelockingentailsguardingclientcodethatusessomeobjectXwiththelockXusestoguarditsown state.InordertouseclientͲsidelocking,youmustknowwhatlockXuses. ThedocumentationforVectorandthesynchronizedwrapperclassesstates,albeitobliquely,thattheysupportclientͲ sidelocking,byusingtheintrinsiclockfortheVectororthewrappercollection(notthewrappedcollection).Listing 4.15showsaputIfAbsentoperationonathreadͲsafeListthatcorrectlyusesclientͲsidelocking. Listing4.15.ImplementingPutǦifǦabsentwithClientǦsideLocking. @ThreadSafe public class ListHelper { public List list = Collections.synchronizedList(new ArrayList()); ... public boolean putIfAbsent(E x) { synchronized (list) { boolean absent = !list.contains(x); if (absent) list.add(x); return absent; } } } Ifextendingaclasstoaddanotheratomicoperationisfragilebecauseitdistributesthelockingcodeforaclassover multipleclassesinanobjecthierarchy,clientͲsidelockingisevenmorefragilebecauseitentailsputtinglockingcodefor classCintoclassesthataretotallyunrelatedtoC.ExercisecarewhenusingclientͲsidelockingonclassesthatdonot committotheirlockingstrategy. ClientͲsidelockinghasalotincommonwithclassextensionͲtheybothcouplethebehaviorofthederivedclasstothe implementationofthebaseclass.Justasextensionviolatesencapsulationofimplementation[EJItem14],clientͲside lockingviolatesencapsulationofsynchronizationpolicy. 4.4.2.Composition Thereisalessfragilealternativeforaddinganatomicoperationtoanexistingclass:composition. ImprovedListin Listing4.16implementstheListoperationsbydelegatingthemtoanunderlyingListinstance,andaddsanatomic putIfAbsentmethod.(LikeCollections.synchronizedListandothercollectionswrappers,ImprovedListassumes thatoncealistispassedtoitsconstructor,theclientwillnotusetheunderlyinglistdirectlyagain,accessingitonly throughtheImprovedList.) ImprovedListaddsanadditionalleveloflockingusingitsownintrinsiclock.Itdoesnotcarewhethertheunderlying LististhreadͲsafe,becauseitprovidesitsownconsistentlockingthatprovidesthreadsafetyeveniftheListisnot threadͲsafe or changes its locking implementation. While the extra layer of synchronization may add some small performance penalty,[7] the implementation in ImprovedList is less fragile than attempting to mimic the locking  49 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ16BChapter4. ComposingObjects strategyofanotherobject.Ineffect,we'veusedtheJavamonitorpatterntoencapsulateanexistingList,andthisis guaranteedtoprovidethreadsafetysolongasourclassholdstheonlyoutstandingreferencetotheunderlyingList. Listing4.16.ImplementingPutǦifǦabsentUsingComposition. @ThreadSafe public class ImprovedList implements List { private final List list; public ImprovedList(List list) { this.list = list; } public synchronized boolean putIfAbsent(T x) { boolean contains = list.contains(x); if (contains) list.add(x); return !contains; } public synchronized void clear() { list.clear(); } // ... similarly delegate other List methods } [7]ThepenaltywillbesmallbecausethesynchronizationontheunderlyingListisguaranteedtobeuncontendedandthereforefast;seeChapter 11. 4.5.DocumentingSynchronizationPolicies Documentationisoneofthemostpowerful(and,sadly,mostunderutilized)toolsformanagingthreadsafety.Users looktothedocumentationtofindoutifaclassisthreadͲsafe,andmaintainerslooktothedocumentationtounderstand theimplementationstrategysotheycanmaintainitwithoutinadvertentlycompromisingsafety.Unfortunately,bothof theseconstituenciesusuallyfindlessinformationinthedocumentationthanthey'dlike. Documentaclass'sthreadsafetyguaranteesforitsclients;documentitssynchronizationpolicyforitsmaintainers. Eachuseofsynchronized,volatile,oranythreadͲsafeclassreflectsasynchronizationpolicydefiningastrategyfor ensuringtheintegrityofdatainthefaceofconcurrentaccess.Thatpolicyisanelementofyourprogram'sdesign,and shouldbedocumented.Ofcourse,thebesttimetodocumentdesigndecisionsisatdesigntime.Weeksormonthslater, thedetailsmaybeablurͲsowriteitdownbeforeyouforget. Craftingasynchronizationpolicyrequiresanumberofdecisions:whichvariablestomakevolatile,whichvariablesto guardwithlocks,whichlock(s)guardwhichvariables,whichvariablestomakeimmutableorconfinetoathread,which operationsmustbeatomic,etc.Someofthesearestrictlyimplementationdetailsandshouldbedocumentedforthe sake of future maintainers, but some affect the publicly observable locking behavior of your class and should be documentedaspartofitsspecification. Attheveryleast,documentthethreadsafetyguaranteesmadebyaclass.IsitthreadͲsafe?Doesitmakecallbackswith alockheld?Arethereanyspecificlocksthataffectitsbehavior?Don'tforceclientstomakeriskyguesses.Ifyoudon't wanttocommittosupportingclientͲsidelocking,that'sfine,butsayso.Ifyouwantclientstobeabletocreatenew atomicoperationsonyourclass,aswedidinSection4.4,youneedtodocumentwhichlockstheyshouldacquiretodo sosafely.Ifyouuselockstoguardstate,documentthisforfuturemaintainers,becauseit'ssoeasyͲthe@GuardedBy annotationwilldothetrick.Ifyouusemoresubtlemeanstomaintainthreadsafety,documentthembecausetheymay notbeobvioustomaintainers. Thecurrentstateofaffairsinthreadsafetydocumentation,evenintheplatformlibraryclasses,isnotencouraging.How manytimeshaveyoulookedattheJavadocforaclassandwonderedwhetheritwasthreadͲsafe?[8]Mostclassesdon't offer any clue either way. Many official Java technology specifications, such as servlets and JDBC, woefully underdocumenttheirthreadsafetypromisesandrequirements. [8]Ifyou'veneverwonderedthis,weadmireyouroptimism. Whileprudencesuggeststhatwenotassumebehaviorsthataren'tpartofthespecification,wehaveworktogetdone, andweareoftenfacedwithachoiceofbadassumptions.ShouldweassumeanobjectisthreadͲsafebecauseitseems thatitoughttobe?ShouldweassumethataccesstoanobjectcanbemadethreadͲsafebyacquiringitslockfirst?(This riskytechniqueworksonlyifwecontrolallthecodethataccessesthatobject;otherwise,itprovidesonlytheillusionof threadsafety.)Neitherchoiceisverysatisfying.  50 JavaConcurrencyInPractice Tomakemattersworse,ourintuitionmayoftenbewrongonwhichclassesare"probablythreadͲsafe"andwhichare not.Asanexample,java.text.SimpleDateFormatisn'tthreadͲsafe,buttheJavadocneglectedtomentionthisuntil JDK 1.4. That this particular class isn't threadͲsafe comes as a surprise to many developers. How many programs mistakenlycreateasharedinstanceofanonͲthreadͲsafeobjectanduseditfrommultiplethreads,unawarethatthis mightcauseerroneousresultsunderheavyload? TheproblemwithSimpleDateFormatcouldbeavoidedbynotassumingaclassisthreadͲsafeifitdoesn'tsayso.Onthe other hand, it is impossible to develop a servletͲbased application without making some pretty questionable assumptionsaboutthethreadsafetyofcontainerͲprovidedobjectslike HttpSession.Don'tmakeyourcustomersor colleagueshavetomakeguesseslikethis. 4.5.1.InterpretingVagueDocumentation Many Java technology specifications are silent, or at least unforthcoming, about thread safety guarantees and requirements for interfaces such as ServletContext, HttpSession, or DataSource.[9] Since these interfaces are implementedbyyourcontainerordatabasevendor,youoftencan'tlookatthecodetoseewhatitdoes.Besides,you don'twanttorelyontheimplementationdetailsofoneparticularJDBCdriverͲyouwanttobecompliantwiththe standardsoyourcodeworksproperlywithanyJDBCdriver.Butthewords"thread"and"concurrent"donotappearat allintheJDBCspecification,andappearfrustratinglyrarelyintheservletspecification.Sowhatdoyoudo? [9]Wefinditparticularlyfrustratingthattheseomissionspersistdespitemultiplemajorrevisionsofthespecifications. Youaregoingtohavetoguess.Onewaytoimprovethequalityofyourguessistointerpretthespecificationfromthe perspectiveofsomeonewhowillimplementit(suchasacontainerordatabasevendor),asopposedtosomeonewho willmerelyuseit.ServletsarealwayscalledfromacontainerͲmanagedthread,anditissafetoassumethatifthereis morethanonesuchthread,thecontainerknowsthis.Theservletcontainermakesavailablecertainobjectsthatprovide servicetomultipleservlets,suchasHttpSessionorServletContext.Sotheservletcontainershouldexpecttohave theseobjectsaccessedconcurrently,sinceithascreatedmultiplethreadsandcalledmethodslikeServlet.service fromthemthatcouldreasonablybeexpectedtoaccesstheServletContext. SinceitisimpossibletoimagineasingleͲthreadedcontextinwhichtheseobjectswouldbeuseful,onehastoassume thattheyhavebeenmadethreadͲsafe,eventhoughthespecificationdoesnotexplicitlyrequirethis.Besides,ifthey requiredclientͲsidelocking,onwhatlockshouldtheclientcodesynchronize?Thedocumentationdoesn'tsay,andit seems absurd to guess. This "reasonable assumption" is further bolstered by the examples in the specification and official tutorials that show how to access ServletContext or HttpSession and do not use any clientͲside synchronization. Ontheotherhand,theobjectsplacedintheServletContextorHttpSessionwithsetAttributeareownedbythe webapplication,nottheservletcontainer.Theservletspecificationdoesnotsuggestanymechanismforcoordinating concurrentaccesstosharedattributes.Soattributesstoredbythecontaineronbehalfofthewebapplicationshouldbe threadͲsafeoreffectivelyimmutable.Ifallthecontainerdidwasstoretheseattributesonbehalfofthewebapplication, anotheroptionwouldbetoensurethattheyareconsistentlyguardedbyalockwhenaccessedfromservletapplication code. But because the container may want to serialize objects in the HttpSession for replication or passivation purposes,andtheservletcontainercan'tpossiblyknowyourlockingprotocol,youshouldmakethemthreadͲsafe. OnecanmakeasimilarinferenceabouttheJDBCDataSourceinterface,whichrepresentsapoolofreusabledatabase connections. A DataSource provides service to an application, and it doesn't make much sense in the context of a singleͲthreadedapplication.Itishardtoimagineausecasethatdoesn'tinvolvecallinggetConnectionfrommultiple threads.And,aswithservlets,theexamplesintheJDBCspecificationdonotsuggesttheneedforanyclientͲsidelocking inthemanycodeexamplesusingDataSource.So,eventhoughthespecificationdoesn'tpromisethatDataSourceis threadͲsafeorrequirecontainervendorstoprovideathreadͲsafeimplementation,bythesame"itwouldbeabsurdifit weren't"argument,wehavenochoicebuttoassumethat DataSource.getConnectiondoesnotrequireadditional clientͲsidelocking. On the other hand, we would not make the same argument about the JDBC Connection objects dispensed by the DataSource,sincethesearenotnecessarilyintendedtobesharedbyotheractivitiesuntiltheyarereturnedtothepool. SoifanactivitythatobtainsaJDBCConnectionspansmultiplethreads,itmusttakeresponsibilityforensuringthat access to the Connection is properly guarded by synchronization. (In most applications, activities that use a JDBC ConnectionareimplementedsoastoconfinetheConnectiontoaspecificthreadanyway.)  51 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks Chapter5.BuildingBlocks ThelastchapterexploredseveraltechniquesforconstructingthreadͲsafeclasses,includingdelegatingthreadsafetyto existingthreadͲsafeclasses.Wherepractical,delegationisoneofthemosteffectivestrategiesforcreatingthreadͲsafe classes:justletexistingthreadͲsafeclassesmanageallthestate. Theplatformlibrariesincludearichsetofconcurrentbuildingblocks,suchasthreadͲsafecollectionsandavarietyof synchronizers that can coordinate the control flow of cooperating threads. This chapter covers the most useful concurrentbuildingblocks,especiallythoseintroducedinJava5.0andJava6,andsomepatternsforusingthemto structureconcurrentapplications. 5.1.SynchronizedCollections Thesynchronizedcollectionclassesinclude Vectorand Hashtable,partoftheoriginalJDK,aswellastheircousins addedinJDK1.2,thesynchronizedwrapperclassescreatedbytheCollections.synchronizedXxxfactorymethods. Theseclassesachievethreadsafetybyencapsulatingtheirstateandsynchronizingeverypublicmethodsothatonlyone threadatatimecanaccessthecollectionstate. 5.1.1.ProblemswithSynchronizedCollections ThesynchronizedcollectionsarethreadͲsafe,butyoumaysometimesneedtouseadditionalclientͲsidelockingtoguard compoundactions.Commoncompoundactionsoncollectionsincludeiteration(repeatedlyfetchelementsuntilthe collection is exhausted), navigation (find the next element after this one according to some order), and conditional operationssuchasputͲifͲabsent(checkifaMaphasamappingforkeyK,andifnot,addthemapping(K,V)).Witha synchronizedcollection,thesecompoundactionsarestilltechnicallythreadͲsafeevenwithoutclientͲsidelocking,but theymaynotbehaveasyoumightexpectwhenotherthreadscanconcurrentlymodifythecollection. Listing5.1showstwomethodsthatoperateonaVector,getLastanddelete-Last,bothofwhicharecheckͲthenͲact sequences.Eachcallssizetodeterminethesizeofthearrayandusestheresultingvaluetoretrieveorremovethelast element. Listing5.1.CompoundActionsonaVectorthatmayProduceConfusingResults. public static Object getLast(Vector list) { int lastIndex = list.size() - 1; return list.get(lastIndex); } public static void deleteLast(Vector list) { int lastIndex = list.size() - 1; list.remove(lastIndex); } Thesemethodsseemharmless,andinasensetheyareͲtheycan'tcorrupttheVector,nomatterhowmanythreads callthemsimultaneously.Butthecallerofthesemethodsmighthaveadifferentopinion.IfthreadAcallsgetLastona Vectorwithtenelements,threadBcallsdeleteLastonthesameVector,andtheoperationsareinterleavedasshown inFigure5.1,getLastthrowsArrayIndexOutOfBoundsException.Betweenthecalltosizeandthesubsequentcallto getingetLast,theVectorshrankandtheindexcomputedinthefirststepisnolongervalid.Thisisperfectlyconsistent withthespecificationofVectoritthrowsanexceptionifaskedforanonexistentelement.Butthisisnotwhatacaller expectsgetLasttodo,eveninthefaceofconcurrentmodification,unlessperhapsthe Vectorwasemptytobegin with. Figure5.1.InterleavingofGetlastandDeletelastthatthrowsArrayIndexOutOfBoundsException.   52 JavaConcurrencyInPractice Because the synchronized collections commit to a synchronization policy that supports clientͲside locking, [1] it is possibletocreatenewoperationsthatareatomicwithrespecttoothercollectionoperationsaslongasweknowwhich locktouse.Thesynchronizedcollectionclassesguardeachmethodwiththelockonthesynchronizedcollectionobject itself. By acquiring the collection lock we can make getLast and deleteLast atomic, ensuring that the size of the Vectordoesnotchangebetweencallingsizeandget,asshowninListing5.2. [1]ThisisdocumentedonlyobliquelyintheJava5.0Javadoc,asanexampleofthecorrectiterationidiom. Theriskthatthesizeofthelistmightchangebetweenacalltosizeandthecorrespondingcalltogetisalsopresent whenweiteratethroughtheelementsofaVectorasshowninListing5.3. ThisiterationidiomreliesonaleapoffaiththatotherthreadswillnotmodifytheVectorbetweenthecallstosizeand get.InasingleͲthreadedenvironment,thisassumptionis perfectlyvalid,butwhenotherthreadsmayconcurrently modifythe Vectoritcanleadtotrouble.Justaswith getLast,ifanotherthreaddeletesanelementwhileyouare iterating through the Vector and the operations are interleaved unluckily, this iteration idiom throws ArrayIndexOutOfBoundsException. Listing5.2.CompoundActionsonVectorUsingClientǦsideLocking. public static Object getLast(Vector list) { synchronized (list) { int lastIndex = list.size() - 1; return list.get(lastIndex); } } public static void deleteLast(Vector list) { synchronized (list) { int lastIndex = list.size() - 1; list.remove(lastIndex); } } Listing5.3.IterationthatmayThrowArrayIndexOutOfBoundsException. for (int i = 0; i < vector.size(); i++) doSomething(vector.get(i)); EventhoughtheiterationinListing5.3canthrowanexception,thisdoesn'tmeanVectorisn'tthreadͲsafe.Thestateof theVectorisstillvalidandtheexceptionisinfactinconformancewithitsspecification.However,thatsomethingas mundaneasfetchingthelastelementoriterationthrowanexceptionisclearlyundesirable. TheproblemofunreliableiterationcanagainbeaddressedbyclientͲsidelocking,atsomeadditionalcosttoscalability. By holding the Vector lock for the duration of iteration, as shown in Listing 5.4, we prevent other threads from modifyingtheVectorwhileweareiteratingit.Unfortunately,wealsopreventotherthreadsfromaccessingitatall duringthistime,impairingconcurrency. Listing5.4.IterationwithClientǦsideLocking. synchronized (vector) { for (int i = 0; i < vector.size(); i++) doSomething(vector.get(i)); } 5.1.2.IteratorsandConcurrentmodificationexception WeuseVectorforthesakeofclarityinmanyofourexamples,eventhoughitisconsidereda"legacy"collectionclass. Butthemore"modern"collectionclassesdonoteliminatetheproblemofcompoundactions.Thestandardwayto iterateaCollectioniswithanIterator,eitherexplicitlyorthroughtheforͲeachloopsyntaxintroducedinJava5.0, butusingiteratorsdoesnotobviatetheneedtolockthecollectionduringiterationifotherthreadscanconcurrently modifyit.Theiteratorsreturnedbythesynchronizedcollectionsarenotdesignedtodealwithconcurrentmodification, andtheyarefailͲfastͲmeaningthatiftheydetectthatthecollectionhaschangedsinceiterationbegan,theythrowthe uncheckedConcurrentModificationException. ThesefailͲfastiteratorsarenotdesignedtobefoolproofͲtheyaredesignedtocatchconcurrencyerrorsona"goodͲ faithͲeffort"basisandthusactonlyasearlyͲwarningindicatorsforconcurrencyproblems.Theyareimplementedby associatingamodificationcountwiththecollection:ifthemodificationcountchangesduringiteration,hasNextornext throwsConcurrentModificationException.However,thischeckisdonewithoutsynchronization,sothereisariskof seeingastalevalueofthemodificationcountandthereforethattheiteratordoesnotrealizeamodificationhasbeen made.Thiswasadeliberatedesigntradeofftoreducetheperformanceimpactoftheconcurrentmodificationdetection code.[2]  53 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks [2]ConcurrentModificationExceptioncanariseinsingleͲthreadedcodeaswell;thishappenswhenobjectsareremovedfromthecollectiondirectly ratherthanthroughIterator.remove. Listing5.5illustratesiteratingacollectionwiththeforͲeachloopsyntax.Internally,javacgeneratescodethatusesan Iterator,repeatedlycallinghasNextandnexttoiteratetheList.JustaswithiteratingtheVector,thewaytoprevent ConcurrentModificationExceptionistoholdthecollectionlockforthedurationoftheiteration. Listing5.5.IteratingaListwithanIterator. List widgetList = Collections.synchronizedList(new ArrayList()); ... // May throw ConcurrentModificationException for (Widget w : widgetList) doSomething(w); Thereareseveralreasons,however,whylockingacollectionduringiterationmaybeundesirable.Otherthreadsthat needtoaccessthecollectionwillblockuntiltheiterationiscomplete;ifthecollectionislargeorthetaskperformedfor eachelementislengthy,theycouldwaitalongtime.Also,ifthecollectionislockedasinListing5.4,doSomethingis beingcalledwithalockheld,whichisariskfactorfordeadlock(seeChapter10).Evenintheabsenceofstarvationor deadlockrisk,lockingcollectionsforsignificantperiodsoftimehurtsapplicationscalability.Thelongeralockisheld,the morelikelyitistobecontended,andifmanythreadsareblockedwaitingforalockthroughputandCPUutilizationcan suffer(seeChapter11). Analternativetolockingthecollectionduringiterationistoclonethecollectionanditeratethecopyinstead.Sincethe clone is threadͲconfined, no other thread can modify it during iteration, eliminating the possibility of ConcurrentModificationException.(Thecollectionstillmustbelockedduringthecloneoperationitself.)Cloningthe collectionhasanobviousperformancecost;whetherthisisafavorabletradeoffdependsonmanyfactorsincludingthe sizeofthecollection,howmuchworkisdoneforeachelement,therelativefrequencyofiterationcomparedtoother collectionoperations,andresponsivenessandthroughputrequirements. 5.1.3.HiddenIterators WhilelockingcanpreventiteratorsfromthrowingConcurrentModificationException,youhavetoremembertouse locking everywhere a shared collection might be iterated. This is trickier than it sounds, as iterators are sometimes hidden, as in HiddenIterator in Listing 5.6. There is no explicit iteration in HiddenIterator, but the code in bold entails iteration just the same. The string concatenation gets turned by the compiler into a call to StringBuilder.append(Object),whichinturninvokesthecollection'stoStringmethodͲandtheimplementationof toStringinthestandardcollectionsiteratesthecollectionandcallstoStringoneachelementtoproduceanicely formattedrepresentationofthecollection'scontents. TheaddTenThingsmethodcouldthrowConcurrentModificationException,becausethecollectionisbeingiterated bytoStringintheprocessofpreparingthedebuggingmessage.Ofcourse,therealproblemisthatHiddenIteratoris notthreadͲsafe;theHiddenIteratorlockshouldbeacquiredbeforeusingsetintheprintlncall,butdebuggingand loggingcodecommonlyneglecttodothis. Thereallessonhereisthatthegreaterthedistancebetweenthestateandthesynchronizationthatguardsit,themore likelythatsomeonewillforgettousepropersynchronizationwhenaccessingthatstate.IfHiddenIteratorwrapped theHashSetwithasynchronizedSet,encapsulatingthesynchronization,thissortoferrorwouldnotoccur. Justasencapsulatinganobject'sstatemakesiteasiertopreserveitsinvariants,encapsulatingitssynchronizationmakes iteasiertoenforceitssynchronizationpolicy.   54 JavaConcurrencyInPractice Listing5.6.IterationHiddenwithinStringConcatenation.Don'tDothis. public class HiddenIterator { @GuardedBy("this") private final Set set = new HashSet(); public synchronized void add(Integer i) { set.add(i); } public synchronized void remove(Integer i) { set.remove(i); } public void addTenThings() { Random r = new Random(); for (int i = 0; i < 10; i++) add(r.nextInt()); System.out.println("DEBUG: added ten elements to " + set); } } Iteration is also indirectly invoked by the collection's hashCode and equals methods, which may be called if the collectionisusedasanelementorkeyofanothercollection.Similarly,thecontainsAll,removeAll,andretainAll methods,aswellastheconstructorsthattakecollectionsarearguments,alsoiteratethecollection.Alloftheseindirect usesofiterationcancauseConcurrentModificationException. 5.2.ConcurrentCollections Java 5.0 improves on the synchronized collections by providing several concurrent collection classes. Synchronized collectionsachievetheirthreadsafetybyserializingallaccesstothecollection'sstate.Thecostofthisapproachispoor concurrency;whenmultiplethreadscontendforthecollectionͲwidelock,throughputsuffers. Theconcurrentcollections,ontheotherhand,aredesignedforconcurrentaccessfrommultiplethreads.Java5.0adds ConcurrentHashMap,areplacementforsynchronizedhashͲbasedMapimplementations,andCopyOnWriteArrayList,a replacement for synchronized List implementations for cases where traversal is the dominant operation. The new ConcurrentMapinterfaceaddssupportforcommoncompoundactionssuchasputͲifͲabsent,replace,andconditional remove. Replacingsynchronizedcollectionswithconcurrentcollectionscanofferdramaticscalabilityimprovementswithlittle risk. Java5.0alsoaddstwonewcollectiontypes,QueueandBlockingQueue.AQueueisintendedtoholdasetofelements temporarilywhiletheyawaitprocessing.Severalimplementationsareprovided,includingConcurrentLinkedQueue,a traditionalFIFOqueue,andPriorityQueue,a(nonconcurrent)priorityorderedqueue.Queueoperationsdonotblock; ifthequeueisempty,theretrievaloperationreturns null.Whileyoucansimulatethebehaviorofa Queuewitha Listinfact,LinkedListalsoimplementsQueueͲtheQueueclasseswereaddedbecauseeliminatingtherandomͲaccess requirementsofListadmitsmoreefficientconcurrentimplementations. BlockingQueue extends Queue to add blocking insertion and retrieval operations. If the queue is empty, a retrieval blocksuntilanelementisavailable,andifthequeueisfull(forboundedqueues)aninsertionblocksuntilthereisspace available. Blocking queues are extremely useful in producerͲconsumer designs, and are covered in greater detail in Section5.3. Just as ConcurrentHashMap is a concurrent replacement for a synchronized hashͲbased Map, Java 6 adds ConcurrentSkipListMap and ConcurrentSkipListSet, which are concurrent replacements for a synchronized SortedMaporSortedSet(suchasTreeMaporTreeSetwrappedwithsynchronizedMap). 5.2.1.ConcurrentHashMap The synchronized collections classes hold a lock for the duration of each operation. Some operations, such as HashMap.getorList.contains,mayinvolvemoreworkthanisinitiallyobvious:traversingahashbucketorlisttofind aspecificobjectentailscallingequals(whichitselfmayinvolveafairamountofcomputation)onanumberofcandidate objects. In a hashͲbased collection, if hashCode does not spread out hash values well, elements may be unevenly distributed among buckets; in the degenerate case, a poor hash function will turn a hash table into a linked list. Traversingalonglistandcallingequalsonsomeoralloftheelementscantakealongtime,andduringthattimeno otherthreadcanaccessthecollection.  55 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks ConcurrentHashMapisahashͲbasedMaplikeHashMap,butitusesanentirelydifferentlockingstrategythatoffersbetter concurrencyandscalability.Insteadofsynchronizingeverymethodonacommonlock,restrictingaccesstoasingle threadatatime,itusesafinerͲgrainedlockingmechanismcalledlockstriping(seeSection11.4.3)toallowagreater degreeofsharedaccess.Arbitrarilymanyreadingthreadscanaccessthemapconcurrently,readerscanaccessthemap concurrentlywithwriters,andalimitednumberofwriterscanmodifythemapconcurrently.Theresultisfarhigher throughputunderconcurrentaccess,withlittleperformancepenaltyforsingleͲthreadedaccess. ConcurrentHashMap, along with the other concurrent collections, further improve on the synchronized collection classesbyprovidingiteratorsthatdonotthrowConcurrentModificationException,thuseliminatingtheneedtolock thecollectionduringiteration.TheiteratorsreturnedbyConcurrentHashMapareweaklyconsistentinsteadoffailͲfast. Aweaklyconsistentiteratorcantolerateconcurrentmodification,traverseselementsastheyexistedwhentheiterator wasconstructed,andmay(butisnotguaranteedto)reflectmodificationstothecollectionaftertheconstructionofthe iterator. Aswithallimprovements,therearestillafewtradeoffs.ThesemanticsofmethodsthatoperateontheentireMap,such assizeandisEmpty,havebeenslightlyweakenedtoreflecttheconcurrentnatureofthecollection.Sincetheresultof sizecouldbeoutofdatebythetimeitiscomputed,itisreallyonlyanestimate,so sizeisallowedtoreturnan approximation instead of an exact count. While at first this may seem disturbing, in reality methods like size and isEmpty are far less useful in concurrent environments because these quantities are moving targets. So the requirements for these operations were weakened to enable performance optimizations for the most important operations,primarilyget,put,containsKey,andremove. TheonefeatureofferedbythesynchronizedMapimplementationsbutnotbyConcurrentHashMapistheabilitytolock themapforexclusiveaccess.WithHashtableandsynchronizedMap,acquiringtheMaplockpreventsanyotherthread fromaccessingit.Thismightbenecessaryinunusualcasessuchasaddingseveralmappingsatomically,oriteratingthe Mapseveraltimesandneedingtoseethesameelementsinthesameorder.Onthewhole,though,thisisareasonable tradeoff:concurrentcollectionsshouldbeexpectedtochangetheircontentscontinuously. BecauseithassomanyadvantagesandsofewdisadvantagescomparedtoHashtableorsynchronizedMap,replacing synchronizedMapimplementationswithConcurrentHashMapinmostcasesresultsonlyinbetterscalability.Onlyifyour application needs to lock the map for exclusive access [3] is ConcurrentHashMap not an appropriate dropͲin replacement. [3]OrifyouarerelyingonthesynchronizationsideeffectsofthesynchronizedMapimplementations. 5.2.2.AdditionalAtomicMapOperations Since a ConcurrentHashMap cannot be locked for exclusive access, we cannot use clientͲside locking to create new atomicoperationssuchasputͲifͲabsent,aswedidforVectorinSection4.4.1.Instead,anumberofcommoncompound operations such as putͲifͲabsent, removeͲifͲequal, and replaceͲifͲequal are implemented as atomic operations and specified by the ConcurrentMap interface, shown in Listing 5.7. If you find yourself adding such functionality to an existingsynchronizedMapimplementation,itisprobablyasignthatyoushouldconsiderusingaConcurrentMapinstead. 5.2.3.CopyOnWriteArrayList CopyOnWriteArrayListisaconcurrentreplacementforasynchronizedListthatoffersbetterconcurrencyinsome common situations and eliminates the need to lock or copy the collection during iteration. (Similarly, CopyOnWriteArraySetisaconcurrentreplacementforasynchronizedSet.) ThecopyͲonͲwritecollectionsderivetheirthreadsafetyfromthefactthataslongasaneffectivelyimmutableobjectis properlypublished,nofurthersynchronizationisrequiredwhenaccessingit.Theyimplementmutabilitybycreatingand republishinganewcopyofthecollectioneverytimeitismodified.IteratorsforthecopyͲonͲwritecollectionsretaina referencetothebackingarraythatwascurrentatthestartofiteration,andsincethiswillneverchange,theyneedto synchronizeonlybrieflytoensurevisibilityofthearraycontents.Asaresult,multiplethreadscaniteratethecollection withoutinterferencefromoneanotherorfromthreadswantingtomodifythecollection.Theiteratorsreturnedbythe copyͲonͲwritecollectionsdonotthrowConcurrentModificationExceptionandreturntheelementsexactlyasthey wereatthetimetheiteratorwascreated,regardlessofsubsequentmodifications.  56 JavaConcurrencyInPractice Listing5.7.ConcurrentMapInterface. public interface ConcurrentMap extends Map { // Insert into map only if no value is mapped from K V putIfAbsent(K key, V value); // Remove only if K is mapped to V boolean remove(K key, V value); // Replace value only if K is mapped to oldValue boolean replace(K key, V oldValue, V newValue); // Replace value only if K is mapped to some value V replace(K key, V newValue); } Obviously, there is some cost to copying the backing array every time the collection is modified, especially if the collectionislarge;thecopyͲonͲwritecollectionsarereasonabletouseonlywheniterationisfarmorecommonthan modification.ThiscriterionexactlydescribesmanyeventͲnotificationsystems:deliveringanotificationrequiresiterating thelistofregisteredlistenersandcallingeachoneofthem,andinmostcasesregisteringorunregisteringanevent listenerisfarlesscommonthanreceivinganeventnotification.(See[CPJ2.4.4]formoreinformationoncopyͲonͲwrite.) 5.3.BlockingQueuesandtheProducerǦconsumerPattern Blockingqueuesprovideblockingputandtakemethodsaswellasthetimedequivalentsofferandpoll.Ifthequeue isfull,putblocksuntilspacebecomesavailable;ifthequeueisempty,takeblocksuntilanelementisavailable.Queues canbeboundedorunbounded;unboundedqueuesareneverfull,soaputonanunboundedqueueneverblocks. Blocking queues support the producerͲconsumer design pattern. A producerͲconsumer design separates the identificationofworkto bedonefromtheexecutionofthat workbyplacingworkitemsona"todo"listforlater processing,ratherthanprocessingthemimmediatelyastheyareidentified.TheproducerͲconsumerpatternsimplifies developmentbecauseitremovescodedependenciesbetweenproducerandconsumerclasses,andsimplifiesworkload managementbydecouplingactivitiesthatmayproduceorconsumedataatdifferentorvariablerates. In a producerͲconsumer design built around a blocking queue, producers place data onto the queue as it becomes available,andconsumersretrievedatafromthequeuewhentheyarereadytotaketheappropriateaction.Producers don'tneedtoknowanythingabouttheidentityornumberofconsumers,orevenwhethertheyaretheonlyproducerͲ alltheyhavetodoisplacedataitemsonthequeue.Similarly,consumersneednotknowwhotheproducersareor where the work came from. BlockingQueue simplifies the implementation of producerͲconsumer designs with any numberofproducersandconsumers.OneofthemostcommonproducerͲconsumerdesignsisathreadpoolcoupled withaworkqueue;thispatternisembodiedintheExecutortaskexecutionframeworkthatisthesubjectofChapters6 and8. ThefamiliardivisionoflaborfortwopeoplewashingthedishesisanexampleofaproducerͲconsumerdesign:one personwashesthedishesandplacestheminthedishrack,andtheotherpersonretrievesthedishesfromtherackand driesthem.Inthisscenario,thedishrackactsasablockingqueue;iftherearenodishesintherack,theconsumerwaits untiltherearedishestodry,andiftherackfillsup,theproducerhastostopwashinguntilthereismorespace.This analogyextendstomultipleproducers(thoughtheremaybecontentionforthesink)andmultipleconsumers;each workerinteractsonlywiththedishrack.Nooneneedstoknowhowmanyproducersorconsumersthereare,orwho producedagivenitemofwork. Thelabels"producer"and"consumer"arerelative;anactivitythatactsasaconsumerinonecontextmayactasa producerinanother.Dryingthedishes"consumes"cleanwetdishesand"produces"cleandrydishes.Athirdperson wantingtohelpmightputawaythedrydishes,inwhichcasethedrierisbothaconsumerandaproducer,andthereare nowtwosharedworkqueues(eachofwhichmayblockthedrierfromproceeding.) Blocking queues simplify the coding of consumers, since take blocks until data is available. If the producers don't generate work fast enough to keep the consumers busy, the consumers just wait until more work is available. Sometimesthisisperfectlyacceptable(asinaserverapplicationwhennoclientisrequestingservice),andsometimesit indicatesthattheratioofproducerthreadstoconsumerthreadsshouldbeadjustedtoachievebetterutilization(asina webcrawlerorotherapplicationinwhichthereiseffectivelyinfiniteworktodo). Iftheproducersconsistentlygenerateworkfasterthantheconsumerscanprocessit,eventuallytheapplicationwillrun outofmemorybecauseworkitemswillqueueupwithoutbound.Again,theblockingnatureofputgreatlysimplifies coding of producers; if we use a bounded queue, then when the queue fills up the producers block, giving the consumerstimetocatchupbecauseablockedproducercannotgeneratemorework.  57 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks Blockingqueuesalsoprovidean offermethod,whichreturnsafailurestatusiftheitemcannotbeenqueued.This enablesyoutocreatemoreflexiblepoliciesfordealingwithoverload,suchassheddingload,serializingexcesswork items and writing them to disk, reducing the number of producer threads, or throttling producers in some other manner. Boundedqueuesareapowerfulresourcemanagementtoolforbuildingreliableapplications:theymakeyourprogram morerobusttooverloadbythrottlingactivitiesthatthreatentoproducemoreworkthancanbehandled. WhiletheproducerͲconsumerpatternenablesproducerandconsumercodetobedecoupledfromeachother,their behaviorisstillcoupledindirectlythroughthesharedworkqueue.Itistemptingtoassumethattheconsumerswill always keep up, so that you need not place any bounds on the size of work queues, but this is a prescription for rearchitectingyoursystemlater.BuildresourcemanagementintoyourdesignearlyusingblockingqueuesͲitisalot easiertodothisupfrontthantoretrofititlater.Blockingqueuesmakethiseasyforanumberofsituations,butif blockingqueuesdon'tfiteasilyintoyourdesign,youcancreateotherblockingdatastructuresusingSemaphore(see Section5.5.3). TheclasslibrarycontainsseveralimplementationsofBlockingQueue.LinkedBlockingQueueandArrayBlockingQueue areFIFOqueues,analogoustoLinkedListandArrayListbutwithbetterconcurrentperformancethanasynchronized List.PriorityBlockingQueueisapriorityͲorderedqueue,whichisusefulwhenyouwanttoprocesselementsinan orderotherthanFIFO.Justlikeothersortedcollections,PriorityBlockingQueuecancompareelementsaccordingto theirnaturalorder(iftheyimplementComparable)orusingaComparator. ThelastBlockingQueueimplementation,SynchronousQueue,isnotreallyaqueueatall,inthatitmaintainsnostorage spaceforqueuedelements.Instead,itmaintainsalistofqueuedthreadswaitingtoenqueueordequeueanelement.In thedishͲwashinganalogy,thiswouldbelikehavingnodishrack,butinsteadhandingthewasheddishesdirectlytothe nextavailabledryer.Whilethismayseemastrangewaytoimplementaqueue,itreducesthelatencyassociatedwith moving data from producer to consumer because the work can be handed off directly. (In a traditional queue, the enqueueanddequeueoperationsmustcompletesequentiallybeforeaunit ofworkcanbehandedoff.)Thedirect handoffalsofeedsbackmoreinformationaboutthestateofthetasktotheproducer;whenthehandoffisaccepted,it knowsaconsumerhastakenresponsibilityforit,ratherthansimplylettingitsitonaqueuesomewhereͲmuchlikethe differencebetweenhandingadocumenttoacolleagueandmerelyputtingitinhermailboxandhopingshegetsitsoon. SinceaSynchronousQueuehasnostoragecapacity,putandtakewillblockunlessanotherthreadisalreadywaitingto participateinthehandoff.Synchronousqueuesaregenerallysuitableonlywhenthereareenoughconsumersthatthere nearlyalwayswillbeonereadytotakethehandoff. 5.3.1.Example:DesktopSearch Onetypeofprogramthatisamenabletodecompositionintoproducersandconsumersisanagentthatscanslocal drivesfordocumentsandindexesthemforlatersearching,similartoGoogleDesktoportheWindowsIndexingservice. DiskCrawlerinListing5.8showsaproducertaskthatsearchesafilehierarchyforfilesmeetinganindexingcriterion andputstheirnamesontheworkqueue;IndexerinListing5.8showstheconsumertaskthattakesfilenamesfromthe queueandindexesthem. The producerͲconsumer pattern offers a threadͲfriendly means of decomposing the desktop search problem into simplercomponents.FactoringfileͲcrawlingandindexingintoseparateactivitiesresultsincodethatismorereadable andreusablethanwithamonolithicactivitythatdoesboth;eachoftheactivitieshasonlyasingletasktodo,andthe blockingqueuehandlesalltheflowcontrol,sothecodeforeachissimplerandclearer. The producerͲconsumer pattern also enables several performance benefits. Producers and consumers can execute concurrently; if one is I/OͲbound and the other is CPUͲbound, executing them concurrently yields better overall throughput than executing them sequentially. If the producer and consumer activities are parallelizable to different degrees,tightlycouplingthemreducesparallelizabilitytothatofthelessparallelizableactivity. Listing5.9startsseveralcrawlersandindexers,eachintheirownthread.Aswritten,theconsumerthreadsneverexit, whichpreventstheprogramfromterminating;weexamineseveraltechniquesforaddressingthisprobleminChapter7. While this example uses explicitly managed threads, many producerͲconsumer designs can be expressed using the Executortaskexecutionframework,whichitselfusestheproducerͲconsumerpattern. 5.3.2.SerialThreadConfinement Theblockingqueueimplementationsinjava.util.concurrentallcontainsufficientinternalsynchronizationtosafely publishobjectsfromaproducerthreadtotheconsumerthread.  58 JavaConcurrencyInPractice Formutableobjects,producerͲconsumerdesignsandblockingqueuesfacilitateserialthreadconfinementforhanding offownershipofobjectsfromproducerstoconsumers.AthreadͲconfinedobjectisownedexclusivelybyasinglethread, butthatownershipcanbe"transferred"bypublishingitsafelywhereonlyoneotherthreadwillgainaccesstoitand ensuringthatthepublishingthreaddoesnotaccessitafterthehandoff.Thesafepublicationensuresthattheobject's stateisvisibletothenewowner,andsincetheoriginalownerwillnottouchitagain,itisnowconfinedtothenew thread.Thenewownermaymodifyitfreelysinceithasexclusiveaccess. Objectpoolsexploitserialthreadconfinement,"lending"anobjecttoarequestingthread.Aslongasthepoolcontains sufficientinternal synchronization to publish the pooled object safely, and as long as the clients do not themselves publishthepooledobjectoruseitafterreturningittothepool,ownershipcanbetransferredsafelyfromthreadto thread. Onecouldalsouseotherpublicationmechanismsfortransferringownershipofamutableobject,butitisnecessaryto ensurethatonlyonethreadreceivestheobjectbeinghandedoff.Blockingqueuesmakethiseasy;withalittlemore work, it could also done with the atomic remove method of ConcurrentMap or the compareAndSet method of AtomicReference. Listing5.8.ProducerandConsumerTasksinaDesktopSearchApplication. public class FileCrawler implements Runnable { private final BlockingQueue fileQueue; private final FileFilter fileFilter; private final File root; ... public void run() { try { crawl(root); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } private void crawl(File root) throws InterruptedException { File[] entries = root.listFiles(fileFilter); if (entries != null) { for (File entry : entries) if (entry.isDirectory()) crawl(entry); else if (!alreadyIndexed(entry)) fileQueue.put(entry); } } } public class Indexer implements Runnable { private final BlockingQueue queue; public Indexer(BlockingQueue queue) { this.queue = queue; } public void run() { try { while (true) indexFile(queue.take()); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } Listing5.9.StartingtheDesktopSearch. public static void startIndexing(File[] roots) { BlockingQueue queue = new LinkedBlockingQueue(BOUND); FileFilter filter = new FileFilter() { public boolean accept(File file) { return true; } }; for (File root : roots) new Thread(new FileCrawler(queue, filter, root)).start(); for (int i = 0; i < N_CONSUMERS; i++) new Thread(new Indexer(queue)).start(); }    59 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks 5.3.3.DequesandWorkStealing Java6alsoaddsanothertwocollectiontypes,Deque(pronounced"deck")andBlockingDeque,thatextendQueueand BlockingQueue.ADequeisadoubleͲendedqueuethatallowsefficientinsertionandremovalfromboththeheadand thetail.ImplementationsincludeArrayDequeandLinkedBlockingDeque. Just as blocking queues lend themselves to the producerͲconsumer pattern, deques lend themselves to a related pattern called work stealing. A producerͲconsumer design has one shared work queue for all consumers; in a work stealingdesign,everyconsumerhasitsowndeque.Ifaconsumerexhauststheworkinitsowndeque,itcanstealwork fromthetailofsomeoneelse'sdeque.WorkstealingcanbemorescalablethanatraditionalproducerͲconsumerdesign becauseworkersdon'tcontendforasharedworkqueue;mostofthetimetheyaccessonlytheirowndeque,reducing contention. When a worker has to access another's queue, it does so from the tail rather than the head, further reducingcontention. WorkstealingiswellsuitedtoproblemsinwhichconsumersarealsoproducersͲwhenperformingaunitofworkis likelytoresultintheidentificationofmorework.Forexample,processingapageinawebcrawlerusuallyresultsinthe identificationofnewpagestobecrawled.Similarly,manygraphͲexploringalgorithms,suchasmarkingtheheapduring garbagecollection,canbeefficientlyparallelizedusingworkstealing.Whenaworkeridentifiesanewunitofwork,it placesitattheendofitsowndeque(oralternatively,inaworksharingdesign,onthatofanotherworker);whenits dequeisempty,itlooksforworkattheendofsomeoneelse'sdeque,ensuringthateachworkerstaysbusy. 5.4.BlockingandInterruptibleMethods Threadsmayblock,orpause,forseveralreasons:waitingforI/Ocompletion,waitingtoacquirealock,waitingtowake upfromThread.sleep,orwaitingfortheresultofacomputationinanotherthread.Whenathreadblocks,itisusually suspended and placed in one of the blocked thread states (BLOCKED, WAITING, or TIMED_WAITING). The distinction betweenablockingoperationandanordinaryoperationthatmerelytakesalongtimetofinishisthatablockedthread mustwaitforaneventthatisbeyonditscontrolbeforeitcanproceedͲtheI/Ocompletes,thelockbecomesavailable, ortheexternalcomputationfinishes.Whenthatexternaleventoccurs,thethreadisplacedbackintheRUNNABLEstate andbecomeseligibleagainforscheduling. The putand takemethodsof BlockingQueuethrowthechecked InterruptedException,asdoanumberofother librarymethodssuchasThread.sleep.WhenamethodcanthrowInterruptedException,itistellingyouthatitisa blockingmethod,andfurtherthatifitisinterrupted,itwillmakeanefforttostopblockingearly. Thread provides the interrupt method for interrupting a thread and for querying whether a thread has been interrupted.Eachthreadhasabooleanpropertythatrepresentsitsinterruptedstatus;interruptingathreadsetsthis status. Interruptionisacooperativemechanism.Onethreadcannotforceanothertostopwhatitisdoinganddosomething else;whenthreadAinterruptsthreadB,AismerelyrequestingthatBstopwhatitisdoingwhenitgetstoaconvenient stoppingpointͲifitfeelslikeit.WhilethereisnothingintheAPIorlanguagespecificationthatdemandsanyspecific applicationͲlevel semantics for interruption, the most sensible use for interruption is to cancel an activity. Blocking methodsthatareresponsivetointerruptionmakeiteasiertocancellongͲrunningactivitiesonatimelybasis. WhenyourcodecallsamethodthatthrowsInterruptedException,thenyourmethodisablockingmethodtoo,and musthaveaplanforrespondingtointerruption.Forlibrarycode,therearebasicallytwochoices: PropagatetheInterruptedException.ThisisoftenthemostsensiblepolicyifyoucangetawaywithitͲjustpropagate theInterruptedExceptiontoyourcaller.ThiscouldinvolvenotcatchingInterruptedException,orcatchingitand throwingitagainafterperformingsomebriefactivityͲspecificcleanup. Restoretheinterrupt.SometimesyoucannotthrowInterruptedException,forinstancewhenyourcodeispartofa Runnable. In these situations, you must catch InterruptedException and restore the interrupted status by calling interrupt on the current thread, so that code higher up the call stack can see that an interrupt was issued, as demonstratedinListing5.10. Youcangetmuchmoresophisticatedwithinterruption,butthesetwoapproachesshouldworkinthevastmajorityof situations.ButthereisonethingyoushouldnotdowithInterruptedExceptioncatchitanddonothinginresponse. Thisdeprivescodehigheruponthecallstackoftheopportunitytoactontheinterruption,becausetheevidencethat thethreadwasinterruptedislost.Theonlysituationinwhichitisacceptabletoswallowaninterruptiswhenyouare extending Thread and therefore control all the code higher up on the call stack. Cancellation and interruption are coveredingreaterdetailinChapter7.  60 JavaConcurrencyInPractice Listing5.10.RestoringtheInterruptedStatussoasNottoSwallowtheInterrupt. public class TaskRunnable implements Runnable { BlockingQueue queue; ... public void run() { try { processTask(queue.take()); } catch (InterruptedException e) { // restore interrupted status Thread.currentThread().interrupt(); } } } 5.5.Synchronizers Blockingqueuesareuniqueamongthecollectionsclasses:notonlydotheyactascontainersforobjects,buttheycan alsocoordinatethecontrolflowofproducerandconsumerthreadsbecausetakeandputblockuntilthequeueenters thedesiredstate(notemptyornotfull). Asynchronizerisanyobjectthatcoordinatesthecontrolflowofthreadsbasedonitsstate.Blockingqueuescanactas synchronizers; other types of synchronizers include semaphores, barriers, and latches. There are a number of synchronizerclassesintheplatformlibrary;ifthesedonotmeetyourneeds,youcanalsocreateyourownusingthe mechanismsdescribedinChapter14. Allsynchronizerssharecertainstructuralproperties:theyencapsulatestatethatdetermineswhetherthreadsarrivingat thesynchronizershouldbeallowedtopassorforcedtowait,providemethodstomanipulatethatstate,andprovide methodstowaitefficientlyforthesynchronizertoenterthedesiredstate. 5.5.1.Latches Alatchisasynchronizerthatcandelaytheprogressofthreadsuntilitreachesitsterminalstate[CPJ3.4.2].Alatchacts asagate:untilthelatchreachestheterminalstatethegateisclosedandnothreadcanpass,andintheterminalstate thegateopens,allowingallthreadstopass.Oncethelatchreachestheterminalstate,itcannotchangestateagain,soit remains open forever. Latches can be used to ensure that certain activities do not proceed until other oneͲtime activitiescomplete,suchas: x Ensuringthatacomputationdoesnotproceeduntilresourcesitneedshavebeeninitialized.Asimplebinary (twoͲstate)latchcouldbeusedtoindicate"ResourceRhasbeeninitialized",andanyactivitythatrequiresR wouldwaitfirstonthislatch. x Ensuringthataservicedoesnotstartuntilotherservicesonwhichitdependshavestarted.Eachservicewould haveanassociatedbinarylatch;startingserviceSwouldinvolvefirstwaitingonthelatchesforotherserviceson whichSdepends,andthenreleasingtheSlatchafterstartupcompletessoanyservicesthatdependonScan thenproceed. x Waitinguntilallthepartiesinvolvedinanactivity,forinstancetheplayersinamultiͲplayergame,arereadyto proceed.Inthiscase,thelatchreachestheterminalstateafteralltheplayersareready. CountDownLatchisaflexiblelatchimplementationthatcanbeusedinanyofthesesituations;itallowsoneormore threads to wait for a set of events to occur. The latch state consists of a counter initialized to a positive number, representingthenumberofeventstowaitfor.ThecountDownmethoddecrementsthecounter,indicatingthatanevent hasoccurred,andthe awaitmethodswaitforthecountertoreachzero,whichhappenswhenalltheeventshave occurred. If the counter is nonzero on entry, await blocks until the counter reaches zero, the waiting thread is interrupted,orthewaittimesout. TestHarnessinListing5.11illustratestwocommonusesforlatches.TestHarnesscreatesanumberofthreadsthatrun agiventaskconcurrently.Itusestwolatches,a"startinggate"andan"endinggate".Thestartinggateisinitializedwith acountofone;theendinggateisinitializedwithacountequaltothenumberofworkerthreads.Thefirstthingeach workerthreaddoesiswaitonthestartinggate;thisensuresthatnoneofthemstartsworkinguntiltheyallarereadyto start.Thelastthingeachdoesiscountdownontheendinggate;thisallowsthemasterthreadtowaitefficientlyuntil thelastoftheworkerthreadshasfinished,soitcancalculatetheelapsedtime. Whydidwebotherwiththelatchesin TestHarnessinsteadofjuststartingthethreadsimmediatelyaftertheyare created?Presumably,wewantedtomeasurehowlongittakestorunataskntimesconcurrently.Ifwesimplycreated andstartedthethreads,thethreadsstartedearlierwouldhavea"headstart"onthelaterthreads,andthedegreeof contentionwouldvaryovertimeasthenumberofactivethreadsincreasedordecreased.Usingastartinggateallows  61 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks themasterthreadtoreleasealltheworkerthreadsatonce,andtheendinggateallowsthemasterthreadtowaitfor thelastthreadtofinishratherthanwaitingsequentiallyforeachthreadtofinish. 5.5.2.FutureTask FutureTask also acts like a latch. (FutureTask implements Future, which describes an abstract resultͲbearing computation[CPJ4.3.3].)Acomputationrepresentedbya FutureTaskisimplementedwitha Callable,theresultͲ bearingequivalentofRunnable,andcanbeinoneofthreestates:waitingtorun,running,orcompleted.Completion subsumesallthewaysacomputationcancomplete,includingnormalcompletion,cancellation,andexception.Oncea FutureTaskentersthecompletedstate,itstaysinthatstateforever. ThebehaviorofFuture.getdependsonthestateofthetask.Ifitiscompleted,getreturnstheresultimmediately,and otherwiseblocksuntilthetasktransitionstothecompletedstateandthenreturnstheresultorthrowsanexception. FutureTaskconveystheresultfromthethreadexecutingthecomputationtothethread(s)retrievingtheresult;the specificationofFutureTaskguaranteesthatthistransferconstitutesasafepublicationoftheresult. Listing5.11.UsingCountDownLatchforStartingandStoppingThreadsinTimingTests. public class TestHarness { public long timeTasks(int nThreads, final Runnable task) throws InterruptedException { final CountDownLatch startGate = new CountDownLatch(1); final CountDownLatch endGate = new CountDownLatch(nThreads); for (int i = 0; i < nThreads; i++) { Thread t = new Thread() { public void run() { try { startGate.await(); try { task.run(); } finally { endGate.countDown(); } } catch (InterruptedException ignored) { } } }; t.start(); } long start = System.nanoTime(); startGate.countDown(); endGate.await(); long end = System.nanoTime(); return end-start; } } FutureTaskisusedbytheExecutorframeworktorepresentasynchronoustasks,andcanalsobeusedtorepresentany potentially lengthy computation that can be started before the results are needed. Preloader in Listing 5.12 uses FutureTasktoperformanexpensivecomputationwhoseresultsareneededlater;bystartingthecomputationearly, youreducethetimeyouwouldhavetowaitlaterwhenyouactuallyneedtheresults.  62 JavaConcurrencyInPractice Listing5.12.UsingFutureTasktoPreloadDatathatisNeededLater. public class Preloader { private final FutureTask future = new FutureTask(new Callable() { public ProductInfo call() throws DataLoadException { return loadProductInfo(); } }); private final Thread thread = new Thread(future); public void start() { thread.start(); } public ProductInfo get() throws DataLoadException, InterruptedException { try { return future.get(); } catch (ExecutionException e) { Throwable cause = e.getCause(); if (cause instanceof DataLoadException) throw (DataLoadException) cause; else throw launderThrowable(cause); } } } PreloadercreatesaFutureTaskthatdescribesthetaskofloadingproductinformationfromadatabaseandathreadin whichthecomputationwillbeperformed.Itprovidesastartmethodtostartthethread,sinceitisinadvisabletostart athreadfromaconstructororstaticinitializer.WhentheprogramlaterneedstheProductInfo,itcancallget,which returnstheloadeddataifitisready,orwaitsfortheloadtocompleteifnot. Tasks described by Callable can throw checked and unchecked exceptions, and any code can throw an Error. Whateverthetaskcode maythrow,itiswrappedinan ExecutionExceptionandrethrownfrom Future.get. This complicatescodethatcalls get,notonlybecauseitmustdealwiththepossibilityof ExecutionException(andthe unchecked CancellationException), but also because the cause of the ExecutionException is returned as a THRowable,whichisinconvenienttodealwith. When getthrowsan ExecutionExceptionin Preloader,thecausewillfallintooneofthreecategories:achecked exceptionthrownbytheCallable,aRuntimeException,oranError.Wemusthandleeachofthesecasesseparately, butwewillusethe launderThrowableutilitymethodinListing5.13toencapsulatesomeofthemessierexceptionͲ handling logic. Before calling launderThrowable, Preloader tests for the known checked exceptions and rethrows them.Thatleavesonlyuncheckedexceptions,whichPreloaderhandlesbycallinglaunderThrowableandthrowingthe result.IftheThrowablepassedtolaunderThrowableisanError,launderThrowablerethrowsitdirectly;ifitisnota RuntimeException,itthrowsanIllegalStateExceptiontoindicatealogicerror.ThatleavesonlyRuntimeException, whichlaunderThrowablereturnstoitscaller,andwhichthecallergenerallyrethrows. Listing5.13.CoercinganUncheckedThrowabletoaRuntimeException. /** If the Throwable is an Error, throw it; if it is a * RuntimeException return it, otherwise throw IllegalStateException */ public static RuntimeException launderThrowable(Throwable t) { if (t instanceof RuntimeException) return (RuntimeException) t; else if (t instanceof Error) throw (Error) t; else throw new IllegalStateException("Not unchecked", t); } 5.5.3.Semaphores Countingsemaphoresareusedtocontrolthenumberofactivitiesthatcanaccessacertainresourceorperformagiven actionatthesametime[CPJ3.4.1].Countingsemaphorescanbeusedtoimplementresourcepoolsortoimposea boundonacollection. ASemaphoremanagesasetofvirtualpermits;theinitialnumberofpermitsispassedtotheSemaphoreconstructor. Activitiescanacquirepermits(aslongassomeremain)andreleasepermitswhentheyaredonewiththem.Ifnopermit isavailable,acquireblocksuntiloneis(oruntilinterruptedortheoperationtimesout).Thereleasemethodreturnsa permittothesemaphore.[4]Adegeneratecaseofacountingsemaphoreisabinarysemaphore,aSemaphorewithan initialcountofone.AbinarysemaphorecanbeusedasamutexwithnonͲreentrantlockingsemantics;whoeverholds thesolepermitholdsthemutex.  63 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks [4]Theimplementationhasnoactualpermitobjects,andSemaphoredoesnotassociatedispensedpermitswiththreads,soapermitacquiredin onethreadcanbereleasedfromanotherthread.Youcanthinkofacquireasconsumingapermitandreleaseascreatingone;aSemaphoreisnot limitedtothenumberofpermitsitwascreatedwith. Semaphoresareusefulforimplementingresourcepoolssuchasdatabaseconnectionpools.Whileitiseasytoconstruct afixedͲsizedpoolthatfailsifyourequestaresourcefromanemptypool,whatyoureallywantistoblockifthepoolis emptyandunblockwhenitbecomesnonemptyagain.IfyouinitializeaSemaphoretothepoolsize,acquireapermit beforetryingtofetcharesourcefromthe pool,and release thepermitafterputtinga resourcebackin the pool, acquireblocksuntilthepoolbecomesnonempty.ThistechniqueisusedintheboundedbufferclassinChapter12.(An easierwaytoconstructablockingobjectpoolwouldbetouseaBlockingQueuetoholdthepooledresources.) Similarly, you can use a Semaphore to turn any collection into a blocking bounded collection, as illustrated by BoundedHashSetinListing5.14.Thesemaphoreisinitializedtothedesiredmaximumsizeofthecollection.Theadd operationacquiresapermitbeforeaddingtheitemintotheunderlyingcollection.Iftheunderlyingaddoperationdoes notactuallyaddanything,itreleasesthepermitimmediately.Similarly,asuccessfulremoveoperationreleasesapermit, enabling more elements to be added. The underlying Set implementation knows nothing about the bound; this is handledbyBoundedHashSet. 5.5.4.Barriers Wehaveseenhowlatchescanfacilitatestartingagroupofrelatedactivitiesorwaitingforagroupofrelatedactivities tocomplete.LatchesaresingleͲuseobjects;oncealatchenterstheterminalstate,itcannotbereset. Barriersaresimilartolatchesinthattheyblockagroupofthreadsuntilsomeeventhasoccurred[CPJ4.4.3].Thekey differenceisthatwithabarrier,allthethreadsmustcometogetheratabarrierpointatthesametimeinorderto proceed.Latchesareforwaitingforevents;barriersareforwaitingforotherthreads.Abarrierimplementstheprotocol somefamiliesusetorendezvousduringadayatthemall:"EveryonemeetatMcDonald'sat6:00;onceyougetthere, staythereuntileveryoneshowsup,andthenwe'llfigureoutwhatwe'redoingnext." CyclicBarrierallowsafixednumberofpartiestorendezvousrepeatedlyatabarrierpointandisusefulinparallel iterativealgorithmsthatbreakdownaproblemintoafixednumberofindependentsubproblems.Threadscallawait whentheyreachthebarrierpoint,andawaitblocksuntilallthethreadshavereachedthebarrierpoint.Ifallthreads meetatthebarrierpoint,thebarrierhasbeensuccessfullypassed,inwhichcaseallthreadsarereleasedandthebarrier isresetsoitcanbeusedagain.Ifacalltoawaittimesoutorathreadblockedinawaitisinterrupted,thenthebarrieris considered broken and all outstanding calls to await terminate with BrokenBarrierException. If the barrier is successfullypassed,awaitreturnsauniquearrivalindexforeachthread,whichcanbeusedto"elect"aleaderthat takessomespecialactioninthenextiteration.CyclicBarrieralsoletsyoupassabarrieractiontotheconstructor; thisisaRunnablethatisexecuted(inoneofthesubtaskthreads)whenthebarrierissuccessfullypassedbutbeforethe blockedthreadsarereleased.  64 JavaConcurrencyInPractice Listing5.14.UsingSemaphoretoBoundaCollection. public class BoundedHashSet { private final Set set; private final Semaphore sem; public BoundedHashSet(int bound) { this.set = Collections.synchronizedSet(new HashSet()); sem = new Semaphore(bound); } public boolean add(T o) throws InterruptedException { sem.acquire(); boolean wasAdded = false; try { wasAdded = set.add(o); return wasAdded; } finally { if (!wasAdded) sem.release(); } } public boolean remove(Object o) { boolean wasRemoved = set.remove(o); if (wasRemoved) sem.release(); return wasRemoved; } } Barriersareoftenusedinsimulations,wheretheworktocalculateonestepcanbedoneinparallelbutallthework associated with a given step must complete before advancing to the next step. For example, in nͲbody particle simulations,eachstepcalculatesanupdatetothepositionofeachparticlebasedonthelocationsandotherattributes oftheotherparticles.Waitingonabarrierbetweeneachupdateensuresthatallupdatesforstepkhavecompleted beforemovingontostepk+1. CellularAutomata in Listing 5.15 demonstrates using a barrier to compute a cellular automata simulation, such as Conway'sLifegame(Gardner,1970).Whenparallelizingasimulation,itisgenerallyimpracticaltoassignaseparate thread to each element (in the case of Life, a cell); this would require too many threads, and the overhead of coordinatingthemwoulddwarfthecomputation.Instead,itmakessensetopartitiontheproblemintoanumberof subparts,leteachthreadsolveasubpart,andthenmergetheresults.CellularAutomatapartitionstheboardintoNcpu parts,whereNcpuisthenumberofCPUsavailable,andassignseachparttoathread.[5]Ateachstep,theworkerthreads calculatenewvaluesforallthecellsintheirpartoftheboard.Whenallworkerthreadshavereachedthebarrier,the barrieractioncommitsthenewvaluestothedatamodel.Afterthebarrieractionruns,theworkerthreadsarereleased tocomputethenextstepofthecalculation,whichincludesconsultinganisDonemethodtodeterminewhetherfurther iterationsarerequired. [5]ForcomputationalproblemslikethisthatdonoI/Oandaccessnoshareddata,NcpuorNcpu+1threadsyieldoptimalthroughput;morethreads donothelp,andmayinfactdegradeperformanceasthethreadscompeteforCPUandmemoryresources. AnotherformofbarrierisExchanger,atwoͲpartybarrierinwhichthepartiesexchangedataatthebarrierpoint[CPJ 3.4.3].Exchangersareusefulwhenthepartiesperformasymmetricactivities,forexamplewhenonethreadfillsabuffer withdataandtheotherthreadconsumesthedatafromthebuffer;thesethreadscoulduseanExchangertomeetand exchange a full buffer for an empty one. When two threads exchange objects via an Exchanger, the exchange constitutesasafepublicationofbothobjectstotheotherparty. Thetimingoftheexchangedependsontheresponsivenessrequirementsoftheapplication.Thesimplestapproachis thatthefillingtaskexchangeswhenthebufferisfull,andtheemptyingtaskexchangeswhenthebufferisempty;this minimizes the number of exchanges but can delay processing of some data if the arrival rate of new data is unpredictable.Anotherapproachwouldbethatthefillerexchangeswhenthebufferisfull,butalsowhenthebufferis partiallyfilledandacertainamountoftimehaselapsed. 5.6.BuildinganEfficient,ScalableResultCache Nearlyeveryserverapplicationusessomeformofcaching.Reusingtheresultsofapreviouscomputationcanreduce latencyandincreasethroughput,atthecostofsomeadditionalmemoryusage. Listing5.15.CoordinatingComputationinaCellularAutomatonwithCyclicBarrier.  65 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks public class CellularAutomata { private final Board mainBoard; private final CyclicBarrier barrier; private final Worker[] workers; public CellularAutomata(Board board) { this.mainBoard = board; int count = Runtime.getRuntime().availableProcessors(); this.barrier = new CyclicBarrier(count, new Runnable() { public void run() { mainBoard.commitNewValues(); }}); this.workers = new Worker[count]; for (int i = 0; i < count; i++) workers[i] = new Worker(mainBoard.getSubBoard(count, i)); } private class Worker implements Runnable { private final Board board; public Worker(Board board) { this.board = board; } public void run() { while (!board.hasConverged()) { for (int x = 0; x < board.getMaxX(); x++) for (int y = 0; y < board.getMaxY(); y++) board.setNewValue(x, y, computeValue(x, y)); try { barrier.await(); } catch (InterruptedException ex) { return; } catch (BrokenBarrierException ex) { return; } } } } public void start() { for (int i = 0; i < workers.length; i++) new Thread(workers[i]).start(); mainBoard.waitForConvergence();} } } Likemanyotherfrequentlyreinventedwheels,cachingoftenlookssimplerthanitis.Anaivecacheimplementationis likely to turn a performance bottleneck into a scalability bottleneck, even if it does improve singleͲthreaded performance.Inthissectionwedevelopanefficientandscalableresultcacheforacomputationallyexpensivefunction. Let'sstartwiththeobviousapproachͲasimpleHashMapandthenlookatsomeofitsconcurrencydisadvantagesand howtofixthem. The Computable interface in Listing 5.16 describes a function with input of type A and result of type V. ExpensiveFunction, which implements Computable, takes a long time to compute its result; we'd like to create a Computablewrapperthatrememberstheresultsofpreviouscomputationsandencapsulatesthecachingprocess.(This techniqueisknownasMemorization.)  66 JavaConcurrencyInPractice Listing5.16.InitialCacheAttemptUsingHashMapandSynchronization. public interface Computable { V compute(A arg) throws InterruptedException; } public class ExpensiveFunction implements Computable { public BigInteger compute(String arg) { // after deep thought... return new BigInteger(arg); } } public class Memorizer1 implements Computable { @GuardedBy("this") private final Map cache = new HashMap(); private final Computable c; public Memorizer1(Computable c) { this.c = c; } public synchronized V compute(A arg) throws InterruptedException { V result = cache.get(arg); if (result == null) { result = c.compute(arg); cache.put(arg, result); } return result; } } Memorizer1inListing5.16showsafirstattempt:usingaHashMaptostoretheresultsofpreviouscomputations.The computemethodfirstcheckswhetherthedesiredresultisalreadycached,andreturnsthepreͲcomputedvalueifitis. Otherwise,theresultiscomputedandcachedintheHashMapbeforereturning. HashMapisnotthreadͲsafe,sotoensurethattwothreadsdonotaccesstheHashMapatthesametime,Memorizer1 takestheconservativeapproachofsynchronizingtheentirecomputemethod.Thisensuresthreadsafetybuthasan obviousscalabilityproblem:onlyonethreadatatimecanexecutecomputeatall.Ifanotherthreadisbusycomputinga result, other threads calling compute may be blocked for a long time. If multiple threads are queued up waiting to computevaluesnotalreadycomputed,computemayactuallytakelongerthanitwouldhavewithoutMemorization. Figure5.2illustrateswhatcouldhappenwhenseveralthreadsattempttouseafunctionmemorizedwiththisapproach. Thisisnotthesortofperformanceimprovementwehadhopedtoachievethroughcaching. Figure5.2.PoorConcurrencyofMemorizer1. Memorizer2inListing5.17improvesontheawfulconcurrentbehaviorofMemorizer1byreplacingtheHashMapwitha ConcurrentHashMap. Since ConcurrentHashMap is threadͲsafe, there is no need to synchronize when accessing the backingMap,thuseliminatingtheserializationinducedbysynchronizingcomputeinMemorizer1.  67 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks Memorizer2 certainly has better concurrent behavior than Memorizer1: multiple threads can actually use it concurrently.ButitstillhassomedefectsasacacheͲthereisawindowofvulnerabilityinwhichtwothreadscalling compute at the same time could end up computing the same value. In the case of memorization, this is merely inefficientͲthe purpose of a cache is to prevent the same data from being calculated multiple times. For a more generalͲpurposecachingmechanism,itisfarworse;foranobjectcachethatissupposedtoprovideonceͲandͲonlyͲonce initialization,thisvulnerabilitywouldalsoposeasafetyrisk. TheproblemwithMemorizer2isthatifonethreadstartsanexpensivecomputation,otherthreadsarenotawarethat the computation is in progress and so may start the same computation, as illustrated in Figure 5.3. We'd like to somehowrepresentthenotionthat"threadXiscurrentlycomputingf(27)",sothatifanotherthreadarriveslookingfor f(27),itknowsthatthemostefficientwaytofinditistoheadovertoThreadX'shouse,hangoutthereuntilXis finished,andthenask"Hey,whatdidyougetforf(27)?" Figure5.3.TwoThreadsComputingtheSameValueWhenUsingMemorizer2. Listing5.17.ReplacingHashMapwithConcurrentHashMap. public class Memorizer2 implements Computable { private final Map cache = new ConcurrentHashMap(); private final Computable c; public Memorizer2(Computable c) { this.c = c; } public V compute(A arg) throws InterruptedException { V result = cache.get(arg); if (result == null) { result = c.compute(arg); cache.put(arg, result); } return result; } } We'vealreadyseenaclassthatdoesalmostexactlythis:FutureTask.FutureTaskrepresentsacomputationalprocess thatmayormaynotalreadyhavecompleted.FutureTask.getreturnstheresultofthecomputationimmediatelyifitis available;otherwiseitblocksuntiltheresulthasbeencomputedandthenreturnsit. Memorizer3 in Listing 5.18 redefines the backing Map for the value cache as a ConcurrentHashMap> insteadofaConcurrentHashMap.Memorizer3firstcheckstoseeiftheappropriatecalculationhasbeenstarted (as opposed to finished, as in Memorizer2). If not, it creates a FutureTask, registers it in the Map, and starts the computation;otherwiseitwaitsfortheresultoftheexistingcomputation.Theresultmightbeavailableimmediatelyor mightbeintheprocessofbeingcomputedͲbutthisistransparenttothecallerofFuture.get. TheMemorizer3implementationisalmostperfect:itexhibitsverygoodconcurrency(mostlyderivedfromtheexcellent concurrencyofConcurrentHashMap),theresultisreturnedefficientlyifitisalreadyknown,andifthecomputationisin progressbyanotherthread,newlyarrivingthreadswaitpatientlyfortheresult.IthasonlyonedefectͲthereisstilla smallwindowofvulnerabilityinwhichtwothreadsmightcomputethesamevalue.Thiswindowisfarsmallerthanin Memorizer2,butbecausetheifblockincomputeisstillanonͲatomiccheckͲthenͲactsequence,itispossiblefortwo threadstocallcomputewiththesamevalueatroughlythesametime,bothseethatthecachedoesnotcontainthe desiredvalue,andbothstartthecomputation.ThisunluckytimingisillustratedinFigure5.4.  68 JavaConcurrencyInPractice Figure5.4.UnluckyTimingthatcouldCauseMemorizer3toCalculatetheSameValueTwice. Listing5.18.MemorizingWrapperUsingFutureTask. public class Memorizer3 implements Computable { private final Map> cache = new ConcurrentHashMap>(); private final Computable c; public Memorizer3(Computable c) { this.c = c; } public V compute(final A arg) throws InterruptedException { Future f = cache.get(arg); if (f == null) { Callable eval = new Callable() { public V call() throws InterruptedException { return c.compute(arg); } }; FutureTask ft = new FutureTask(eval); f = ft; cache.put(arg, ft); ft.run(); // call to c.compute happens here } try { return f.get(); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } } Memorizer3isvulnerabletothisproblembecauseacompoundaction(putͲifͲabsent)isperformedonthebackingmap that cannot be made atomic using locking. Memorizer in Listing 5.19 takes advantage of the atomic putIfAbsent methodofConcurrentMap,closingthewindowofvulnerabilityinMemorizer3. Cachinga Futureinsteadofavaluecreatesthepossibilityofcachepollution:ifacomputationiscancelledorfails, futureattemptstocomputetheresultwillalsoindicatecancellationorfailure.Toavoidthis,Memorizerremovesthe Futurefromthecacheifitdetectsthatthecomputationwascancelled;itmightalsobedesirabletoremovetheFuture upondetectingaRuntimeExceptionifthecomputationmightsucceedonafutureattempt.Memorizeralsodoesnot address cache expiration, but this could be accomplished by using a subclass of FutureTask that associates an expirationtimewitheachresultandperiodicallyscanningthecacheforexpiredentries.(Similarly,itdoesnotaddress cacheeviction,whereoldentriesareremovedtomakeroomfornewonessothatthecachedoesnotconsumetoo muchmemory.) With our concurrent cache implementation complete, we can now add real caching to the factorizing servlet from Chapter2,aspromised.FactorizerinListing5.20usesMemorizertocachepreviouslycomputedvaluesefficientlyand scalably.  69 5288B5287B5286B5249B5230B5229B5200B5148B4916B4722B4721B4720B4719B4594B4593B4569B4568B4567B45 66B4565B4564B4427B4426B4410B4409B4345B4218B4217B4216B4215B3636B3635B3469B3468B3465B3455B3454 B3453B3449B3221B3220B3219B3214B3059B3058B3057B2535B2534B2190B2189B2188B2Ͳ17BChapter5.Building Blocks Listing5.19.FinalImplementationofMemorizer. public class Memorizer implements Computable { private final ConcurrentMap> cache = new ConcurrentHashMap>(); private final Computable c; public Memorizer(Computable c) { this.c = c; } public V compute(final A arg) throws InterruptedException { while (true) { Future f = cache.get(arg); if (f == null) { Callable eval = new Callable() { public V call() throws InterruptedException { return c.compute(arg); } }; FutureTask ft = new FutureTask(eval); f = cache.putIfAbsent(arg, ft); if (f == null) { f = ft; ft.run(); } } try { return f.get(); } catch (CancellationException e) { cache.remove(arg, f); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } } } Listing5.20.FactorizingServletthatCachesResultsUsingMemorizer. @ThreadSafe public class Factorizer implements Servlet { private final Computable c = new Computable() { public BigInteger[] compute(BigInteger arg) { return factor(arg); } }; private final Computable cache = new Memorizer(c); public void service(ServletRequest req, ServletResponse resp) { try { BigInteger i = extractFromRequest(req); encodeIntoResponse(resp, cache.compute(i)); } catch (InterruptedException e) { encodeError(resp, "factorization interrupted"); } } } SummaryofPartI We'vecoveredalotofmaterialsofar!Thefollowing"concurrencycheatsheet"summarizesthemainconceptsand rulespresentedinPartI. x It'sthemutablestate,stupid.[1]  Allconcurrencyissuesboildowntocoordinatingaccesstomutablestate.Thelessmutablestate,theeasieritisto ensurethreadsafety. x Makefieldsfinalunlesstheyneedtobemutable. x ImmutableobjectsareautomaticallythreadͲsafe.  Immutableobjectssimplifyconcurrentprogrammingtremendously.Theyaresimplerandsafer,andcanbeshared freelywithoutlockingordefensivecopying. x Encapsulationmakesitpracticaltomanagethecomplexity.  70 JavaConcurrencyInPractice  You could write a threadͲsafe program with all data stored in global variables, but why would you want to? Encapsulatingdatawithinobjectsmakesiteasiertopreservetheirinvariants;encapsulatingsynchronizationwithin objectsmakesiteasiertocomplywiththeirsynchronizationpolicy. x Guardeachmutablevariablewithalock. x Guardallvariablesinaninvariantwiththesamelock. x Holdlocksforthedurationofcompoundactions. x Aprogramthataccessesamutablevariablefrommultiplethreadswithoutsynchronizationisabrokenprogram. x Don'trelyoncleverreasoningaboutwhyyoudon'tneedtosynchronize. x IncludethreadsafetyinthedesignprocessorexplicitlydocumentthatyourclassisnotthreadͲsafe. x Documentyoursynchronizationpolicy. [1]Duringthe1992U.S.presidentialelection,electoralstrategistJamesCarvillehungasigninBillClinton'scampaignheadquartersreading"The economy,stupid",tokeepthecampaignonmessage.  715BPartII:StructuringConcurrentApplications Ͳ17BChapter5.BuildingBlocks PartII:StructuringConcurrentApplications Chapter6.TaskExecution Chapter7.CancellationandShutdown Chapter8.ApplyingThreadPools Chapter9.GUIApplications  72 JavaConcurrencyInPractice Chapter6.TaskExecution Mostconcurrentapplicationsareorganizedaroundtheexecutionoftasks:abstract,discreteunitsofwork.Dividingthe work of an application into tasks simplifies program organization, facilitates error recovery by providing natural transactionboundaries,andpromotesconcurrencybyprovidinganaturalstructureforparallelizingwork. 6.1.ExecutingTasksinThreads Thefirststepinorganizingaprogramaroundtaskexecutionisidentifyingsensibletaskboundaries.Ideally,tasksare independent activities: work that doesn't depend on the state, result, or side effects of other tasks. Independence facilitatesconcurrency,asindependenttaskscanbeexecutedinparallelifthereareadequateprocessingresources.For greater flexibility in scheduling and load balancing tasks, each task should also represent a small fraction of your application'sprocessingcapacity. Server applications should exhibit both good throughput and good responsiveness under normal load. Application providerswantapplicationstosupportasmanyusersaspossible,soastoreduceprovisioningcostsperuser;users want to get their response quickly. Further, applications should exhibit graceful degradation as they become overloaded,ratherthansimplyfallingoverunderheavyload.Choosinggoodtaskboundaries,coupledwithasensible taskexecutionpolicy(seeSection6.2.2),canhelpachievethesegoals. Mostserverapplicationsofferanaturalchoiceoftaskboundary:individualclientrequests.Webservers,mailservers, fileservers,EJBcontainers,anddatabaseserversallacceptrequestsvianetworkconnectionsfromremoteclients.Using individualrequestsastaskboundariesusuallyoffersbothindependenceandappropriatetasksizing.Forexample,the resultofsubmittingamessagetoamailserverisnotaffectedbytheothermessagesbeingprocessedatthesametime, andhandlingasinglemessageusuallyrequiresaverysmallpercentageoftheserver'stotalcapacity. 6.1.1.ExecutingTasksSequentially Thereareanumberofpossiblepoliciesforschedulingtaskswithinanapplication,someofwhichexploitthepotential forconcurrencybetterthanothers.Thesimplestistoexecutetaskssequentiallyinasinglethread.SingleThreadWeb- ServerinListing6.1processesitstasksͲHTTPrequestsarrivingonport80Ͳsequentially.Thedetailsoftherequest processingaren'timportant;we'reinterestedincharacterizingtheconcurrencyofvariousschedulingpolicies. Listing6.1.SequentialWebServer. class SingleThreadWebServer { public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { Socket connection = socket.accept(); handleRequest(connection); } } } SingleThreadedWebServerissimpleandtheoreticallycorrect,butwouldperformpoorlyinproductionbecauseitcan handle only one request at a time. The main thread alternates between accepting connections and processing the associated request. While the server is handling a request, new connections must wait until it finishes the current request and calls accept again. This might work if request processing were so fast that handleRequest effectively returnedimmediately,butthisdoesn'tdescribeanywebserverintherealworld. Processing a web request involves a mix of computation and I/O. The server must perform socket I/O to read the request and write the response, which can block due to network congestion or connectivity problems. It may also performfileI/Oormakedatabaserequests,whichcanalsoblock.InasingleͲthreadedserver,blockingnotonlydelays completingthecurrentrequest,butpreventspendingrequestsfrombeingprocessedatall.Ifonerequestblocksforan unusuallylongtime,usersmightthinktheserverisunavailablebecauseitappearsunresponsive.Atthesametime, resourceutilizationispoor,sincetheCPUsitsidlewhilethesinglethreadwaitsforitsI/Otocomplete. Inserverapplications,sequentialprocessingrarelyprovideseithergoodthroughputorgoodresponsiveness.Thereare exceptionsͲsuchaswhentasksarefewandlongͲlived,orwhentheserverservesasingleclientthatmakesonlyasingle requestatatimeͲbutmostserverapplicationsdonotworkthisway.[1]  735BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution [1]Insomesituations,sequentialprocessingmayofferasimplicityorsafetyadvantage;mostGUIframeworksprocesstaskssequentiallyusinga singlethread.WereturntothesequentialmodelinChapter9. 6.1.2.ExplicitlyCreatingThreadsforTasks A more responsive approach is to create a new thread for servicing each request, as shown in ThreadPerTaskWebServerinListing6.2. Listing6.2.WebServerthatStartsaNewThreadforEachRequest. class ThreadPerTaskWebServer { public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; new Thread(task).start(); } } } ThreadPerTaskWebServer is similar in structure to the singleͲthreaded versionͲthe main thread still alternates betweenacceptinganincomingconnectionanddispatchingtherequest.Thedifferenceisthatforeachconnection,the mainloopcreatesanewthreadtoprocesstherequestinsteadofprocessingitwithinthemainthread.Thishasthree mainconsequences: x Task processing is offloaded from the main thread, enabling the main loop to resume waiting for the next incoming connection more quickly. This enables new connections to be accepted before previous requests complete,improvingresponsiveness. x Taskscanbeprocessedinparallel,enablingmultiplerequeststobeservicedsimultaneously.Thismayimprove throughputiftherearemultipleprocessors,oriftasksneedtoblockforanyreasonsuchasI/Ocompletion,lock acquisition,orresourceavailability. x TaskͲhandlingcodemustbethreadͲsafe,becauseitmaybeinvokedconcurrentlyformultipletasks. Underlighttomoderateload,thethreadͲperͲtaskapproachisanimprovementoversequentialexecution.Aslongas the request arrival rate does not exceed the server's capacity to handle requests, this approach offers better responsivenessandthroughput. 6.1.3.DisadvantagesofUnboundedThreadCreation For production use, however, the threadͲperͲtask approach has some practical drawbacks, especially when a large numberofthreadsmaybecreated: Threadlifecycleoverhead.Threadcreationandteardownarenotfree.Theactualoverheadvariesacrossplatforms,but threadcreationtakestime,introducinglatencyintorequestprocessing,andrequiressomeprocessingactivitybythe JVMandOS.Ifrequestsarefrequentandlightweight,asinmostserverapplications,creatinganewthreadforeach requestcanconsumesignificantcomputingresources. Resourceconsumption.Activethreadsconsumesystemresources,especiallymemory.Whentherearemorerunnable threads than available processors, threads sit idle. Having many idle threads can tie up a lot of memory, putting pressureonthegarbagecollector,andhavingmanythreadscompetingfortheCPUscanimposeotherperformance costsaswell.IfyouhaveenoughthreadstokeepalltheCPUsbusy,creatingmorethreadswon'thelpandmayeven hurt. Stability.Thereisalimitonhowmanythreadscanbecreated.Thelimitvariesbyplatformandisaffectedbyfactors includingJVMinvocationparameters,therequestedstacksizeintheThreadconstructor,andlimitsonthreadsplaced bytheunderlyingoperatingsystem.[2]Whenyouhitthislimit,themostlikelyresultisanOutOfMemoryError.tryingto recoverfromsuchanerrorisveryrisky;itisfareasiertostructureyourprogramtoavoidhittingthislimit. [2]On32Ͳbitmachines,amajorlimitingfactorisaddressspaceforthreadstacks.Eachthreadmaintainstwoexecutionstacks,oneforJavacode andonefornativecode.TypicalJVMdefaultsyieldacombinedstacksizeofaroundhalfamegabyte.(Youcanchangethiswiththe-XssJVMflag orthroughtheThreadconstructor.)IfyoudividetheperͲthreadstacksizeinto232,yougetalimitofafewthousandsortensofthousandsof threads.Otherfactors,suchasOSlimitations,mayimposestricterlimits.  74 JavaConcurrencyInPractice Uptoacertainpoint,morethreadscanimprovethroughput,butbeyondthatpointcreatingmorethreadsjustslows downyourapplication,andcreatingonethreadtoomanycancauseyourentireapplicationtocrashhorribly.Theway tostayoutofdangeristoplacesomeboundonhowmanythreadsyourapplicationcreates,andtotestyourapplication thoroughlytoensurethat,evenwhenthisboundisreached,itdoesnotrunoutofresources. The problem with the threadͲperͲtask approach is that nothing places any limit on the number of threads created except the rate at which remote users can throw HTTP requests at it. Like other concurrency hazards, unbounded threadcreationmayappeartoworkjustfineduringprototypinganddevelopment,withproblemssurfacingonlywhen theapplicationisdeployedandunderheavyload.Soamalicioususer,orenoughordinaryusers,canmakeyourweb servercrashifthetrafficloadeverreachesacertainthreshold.Foraserverapplicationthatissupposedtoprovidehigh availabilityandgracefuldegradationunderload,thisisaseriousfailing. 6.2.TheExecutorFramework Tasksarelogicalunitsofwork,andthreadsareamechanismbywhichtaskscanrunasynchronously.We'veexamined twopoliciesforexecutingtasksusingthreadsͲexecutetaskssequentiallyinasinglethread,andexecuteeachtaskinits ownthread.Bothhaveseriouslimitations:thesequentialapproachsuffersfrompoorresponsivenessandthroughput, andthethreadͲperͲtaskapproachsuffersfrompoorresourcemanagement. InChapter5,wesawhowtouseboundedqueuestopreventanoverloadedapplicationfromrunningoutofmemory. Threadpoolsofferthesamebenefitforthreadmanagement,andjava.util.concurrentprovidesaflexiblethread poolimplementationaspartoftheExecutorframework.TheprimaryabstractionfortaskexecutionintheJavaclass librariesisnotThread,butExecutor,showninListing6.3. Listing6.3.ExecutorInterface. public interface Executor { void execute(Runnable command); } Executormaybeasimpleinterface,butitformsthebasisforaflexibleandpowerfulframeworkforasynchronoustask execution that supports a wide variety of task execution policies. It provides a standard means of decoupling task submissionfromtaskexecution,describingtaskswithRunnable.TheExecutorimplementationsalsoprovidelifecycle supportandhooksforaddingstatisticsgathering,applicationmanagement,andmonitoring. ExecutorisbasedontheproducerͲconsumerpattern,whereactivitiesthatsubmittasksaretheproducers(producing unitsofworktobedone)andthethreadsthatexecutetasksaretheconsumers(consumingthoseunitsofwork).Using anExecutorisusuallytheeasiestpathtoimplementingaproducerͲconsumerdesigninyourapplication. 6.2.1.Example:WebServerUsingExecutor BuildingawebserverwithanExecutoriseasy.TaskExecutionWebServerinListing6.4replacesthehardͲcodedthread creationwithanExecutor.Inthiscase,weuseoneofthestandardExecutorimplementations,afixedͲsizethreadpool with100threads. In TaskExecutionWebServer, submission of the requestͲhandling task is decoupled from its execution using an Executor, and its behavior can be changed merely by substituting a different Executor implementation. Changing Executorimplementationsorconfigurationisfarlessinvasivethanchangingthewaytasksaresubmitted;Executor configurationisgenerallyaoneͲtimeeventandcaneasilybeexposedfordeploymentͲtimeconfiguration,whereastask submissioncodetendstobestrewnthroughouttheprogramandhardertoexpose.  755BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution Listing6.4.WebServerUsingaThreadPool. class TaskExecutionWebServer { private static final int NTHREADS = 100; private static final Executor exec = Executors.newFixedThreadPool(NTHREADS); public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80); while (true) { final Socket connection = socket.accept(); Runnable task = new Runnable() { public void run() { handleRequest(connection); } }; exec.execute(task); } } } WecaneasilymodifyTaskExecutionWebServertobehavelikeThreadPer-TaskWebServerbysubstitutinganExecutor thatcreatesanewthreadforeachrequest.WritingsuchanExecutoristrivial,asshowninThreadPerTaskExecutorin Listing6.5. Listing6.5.ExecutorthatStartsaNewThreadforEachTask. public class ThreadPerTaskExecutor implements Executor { public void execute(Runnable r) { new Thread(r).start(); }; } Similarly, it is also easy to write an Executor that would make TaskExecutionWebServer behave like the singleͲ threaded version, executing each task synchronously before returning from execute, as shown in WithinThreadExecutorinListing6.6. 6.2.2.ExecutionPolicies Thevalueofdecouplingsubmissionfromexecutionisthatitletsyoueasilyspecify,andsubsequentlychangewithout greatdifficulty,theexecutionpolicyforagivenclassoftasks.Anexecutionpolicyspecifiesthe"what,where,when,and how"oftaskexecution,including: Listing6.6.ExecutorthatExecutesTasksSynchronouslyintheCallingThread. public class WithinThreadExecutor implements Executor { public void execute(Runnable r) { r.run(); }; } x Inwhatthreadwilltasksbeexecuted? x Inwhatordershouldtasksbeexecuted(FIFO,LIFO,priorityorder)? x Howmanytasksmayexecuteconcurrently? x Howmanytasksmaybequeuedpendingexecution? x Ifataskhastoberejectedbecausethesystemisoverloaded,whichtaskshouldbeselectedasthevictim,and howshouldtheapplicationbenotified? x Whatactionsshouldbetakenbeforeorafterexecutingatask? Execution policies are a resource management tool, and the optimal policy depends on the available computing resourcesandyourqualityͲofͲservicerequirements.Bylimitingthenumberofconcurrenttasks,youcanensurethatthe application does not fail due to resource exhaustion or suffer performance problems due to contention for scarce resources.[3] Separating the specification of execution policy from task submission makes it practical to select an executionpolicyatdeploymenttimethatismatchedtotheavailablehardware. [3]Thisisanalogoustooneoftherolesofatransactionmonitorinanenterpriseapplication:itcanthrottletherateatwhichtransactionsare allowedtoproceedsoasnottoexhaustoroverstresslimitedresources. Wheneveryouseecodeoftheform: new Thread(runnable).start()   76 JavaConcurrencyInPractice andyouthinkyoumightatsomepointwantamoreflexibleexecutionpolicy,seriouslyconsiderreplacingitwiththeuse ofanExecutor. 6.2.3.ThreadPools Athreadpool,asitsnamesuggests,managesahomogeneouspoolofworkerthreads.Athreadpoolistightlyboundto aworkqueueholdingtaskswaitingtobeexecuted.Workerthreadshaveasimplelife:requestthenexttaskfromthe workqueue,executeit,andgobacktowaitingforanothertask. ExecutingtasksinpoolthreadshasanumberofadvantagesoverthethreadͲperͲtaskapproach.Reusinganexisting threadinsteadofcreatinganewoneamortizesthreadcreationandteardowncostsovermultiplerequests.Asanadded bonus,sincetheworkerthreadoftenalreadyexistsatthetimetherequestarrives,thelatencyassociatedwiththread creationdoesnotdelaytaskexecution,thusimprovingresponsiveness.Byproperlytuningthesizeofthethreadpool, youcanhaveenoughthreadstokeeptheprocessorsbusywhilenothavingsomanythatyourapplicationrunsoutof memoryorthrashesduetocompetitionamongthreadsforresources. Theclasslibraryprovidesaflexiblethreadpoolimplementationalongwithsomeusefulpredefinedconfigurations.You cancreateathreadpoolbycallingoneofthestaticfactorymethodsinExecutors: newFixedThreadPool.AfixedͲsizethreadpoolcreatesthreadsastasksaresubmitted,uptothemaximumpoolsize, and then attempts to keep the pool size constant (adding new threads if a thread dies due to an unexpected Exception). newCachedThreadPool.Acachedthreadpoolhasmoreflexibilitytoreapidlethreadswhenthecurrentsizeofthepool exceedsthedemandforprocessing,andtoaddnewthreadswhendemandincreases,butplacesnoboundsonthesize ofthepool. newSingleThreadExecutor.AsingleͲthreadedexecutorcreatesasingleworkerthreadtoprocesstasks,replacingitifit diesunexpectedly.Tasksareguaranteedtobeprocessedsequentiallyaccordingtotheorderimposedbythetaskqueue (FIFO,LIFO,priorityorder).[4] [4]SingleͲthreadedexecutorsalsoprovidesufficientinternalsynchronizationtoguaranteethatanymemorywritesmadebytasksarevisibleto subsequenttasks;thismeansthatobjectscanbesafelyconfinedtothe"taskthread"eventhoughthatthreadmaybereplacedwithanotherfrom timetotime. newScheduledThreadPool.AfixedͲsizethreadpoolthatsupportsdelayedandperiodictaskexecution,similartoTimer. (SeeSection6.2.5.) The newFixedThreadPool and newCachedThreadPool factories return instances of the generalͲpurpose ThreadPoolExecutor,whichcanalsobeuseddirectlytoconstructmorespecializedexecutors.Wediscussthreadpool configurationoptionsindepthinChapter8. ThewebserverinTaskExecutionWebServerusesanExecutorwithaboundedpoolofworkerthreads.Submittinga taskwithexecuteaddsthetasktotheworkqueue,andtheworkerthreadsrepeatedlydequeuetasksfromthework queueandexecutethem. SwitchingfromathreadͲperͲtaskpolicytoapoolͲbasedpolicyhasabigeffectonapplicationstability:thewebserver willnolongerfailunderheavyload.[5]Italsodegradesmoregracefully,sinceitdoesnotcreatethousandsofthreads thatcompeteforlimitedCPUandmemoryresources.AndusinganExecutoropensthedoortoallsortsofadditional opportunities for tuning, management, monitoring, logging, error reporting, and other possibilities that would have beenfarmoredifficulttoaddwithoutataskexecutionframework. [5]Whiletheservermaynotfailduetothecreationoftoomanythreads,ifthetaskarrivalrateexceedsthetaskservicerateforlongenoughitis stillpossible(justharder)torunoutofmemorybecauseofthegrowingqueueofRunnablesawaitingexecution.Thiscanbeaddressedwithin theExecutorframeworkbyusingaboundedworkqueueͲseeSection8.3.2. 6.2.4.ExecutorLifecycle We'veseenhowtocreateanExecutorbutnothowtoshutonedown.AnExecutorimplementationislikelytocreate threadsforprocessingtasks.ButtheJVMcan'texituntilallthe(nonͲdaemon)threadshaveterminated,sofailingto shutdownanExecutorcouldpreventtheJVMfromexiting. BecauseanExecutorprocessestasksasynchronously,atanygiventimethestateofpreviouslysubmittedtasksisnot immediatelyobvious.Somemayhavecompleted,somemaybecurrentlyrunning,andothersmaybequeuedawaiting execution.Inshuttingdownanapplication,thereisaspectrumfromgracefulshutdown(finishwhatyou'vestartedbut don't accept any new work) to abrupt shutdown (turn off the power to the machine room), and various points in  775BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution between.SinceExecutorsprovideaservicetoapplications,theyshouldbeabletobeshutdownaswell,bothgracefully andabruptly,andfeedbackinformationtotheapplicationaboutthestatusoftasksthatwereaffectedbytheshutdown. Toaddresstheissueofexecutionservicelifecycle,theExecutorServiceinterfaceextendsExecutor,addinganumber of methods for lifecycle management (as well as some convenience methods for task submission). The lifecycle managementmethodsofExecutorServiceareshowninListing6.7. Listing6.7.LifecycleMethodsinExecutorService. public interface ExecutorService extends Executor { void shutdown(); List shutdownNow(); boolean isShutdown(); boolean isTerminated(); boolean awaitTermination(long timeout, TimeUnit unit) throws InterruptedException; // ... additional convenience methods for task submission } The lifecycle implied by ExecutorService has three statesͲrunning, shutting down, and terminated. ExecutorServicesareinitiallycreatedintherunningstate.Theshutdownmethodinitiatesagracefulshutdown:no newtasksareacceptedbutpreviouslysubmittedtasksareallowedtocomplete Ͳincludingthosethathavenotyet begunexecution.TheshutdownNowmethodinitiatesanabruptshutdown:itattemptstocanceloutstandingtasksand doesnotstartanytasksthatarequeuedbutnotbegun. TaskssubmittedtoanExecutorServiceafterithasbeenshutdownarehandledbytherejectedexecutionhandler(see Section 8.3.3), which might silently discard the task or might cause execute to throw the unchecked RejectedExecutionException. Once all tasks have completed, the ExecutorService transitions to the terminated state.YoucanwaitforanExecutorServicetoreachtheterminatedstatewithawaitTermination,orpollforwhether ithasyetterminatedwithisTerminated.ItiscommontofollowshutdownimmediatelybyawaitTermination,creating the effect of synchronously shutting down the ExecutorService.(Executor shutdown and task cancellation are coveredinmoredetailinChapter7.) LifecycleWebServerinListing6.8extendsourwebserverwithlifecyclesupport.Itcanbeshutdownintwoways: programmaticallybycallingstop,andthroughaclientrequestbysendingthewebserveraspeciallyformattedHTTP request. Listing6.8.WebServerwithShutdownSupport. class LifecycleWebServer { private final ExecutorService exec = ...; public void start() throws IOException { ServerSocket socket = new ServerSocket(80); while (!exec.isShutdown()) { try { final Socket conn = socket.accept(); exec.execute(new Runnable() { public void run() { handleRequest(conn); } }); } catch (RejectedExecutionException e) { if (!exec.isShutdown()) log("task submission rejected", e); } } } public void stop() { exec.shutdown(); } void handleRequest(Socket connection) { Request req = readRequest(connection); if (isShutdownRequest(req)) stop(); else dispatchRequest(req); } } 6.2.5.DelayedandPeriodicTasks TheTimerfacilitymanagestheexecutionofdeferred("runthistaskin100ms")andperiodic("runthistaskevery10 ms") tasks. However, Timer has some drawbacks, and ScheduledThreadPoolExecutor should be thought of as its replacement.[6] You can construct a ScheduledThreadPoolExecutor through its constructor or through the newScheduledThreadPoolfactory.  78 JavaConcurrencyInPractice [6]Timerdoeshavesupportforschedulingbasedonabsolute,notrelativetime,sothattaskscanbesensitivetochangesinthesystemclock; ScheduledThreadPoolExecutorsupportsonlyrelativetime. ATimercreatesonlyasinglethreadforexecutingtimertasks.Ifatimertasktakestoolongtorun,thetimingaccuracy ofotherTimerTaskscansuffer.IfarecurringTimerTaskisscheduledtorunevery10msandanotherTimer-Tasktakes 40mstorun,therecurringtaskeither(dependingonwhetheritwasscheduledatfixedrateorfixeddelay)getscalled fourtimesinrapidsuccessionafterthelongͲrunningtaskcompletes,or"misses"fourinvocationscompletely.Scheduled threadpoolsaddressthislimitationbylettingyouprovidemultiplethreadsforexecutingdeferredandperiodictasks. AnotherproblemwithTimeristhatitbehavespoorlyifaTimerTaskthrowsanuncheckedexception.TheTimerthread doesn'tcatchtheexception,soanuncheckedexceptionthrownfromaTimerTaskterminatesthetimerthread.Timer alsodoesn'tresurrectthethreadinthissituation;instead,iterroneouslyassumestheentireTimerwascancelled.Inthis case,TimerTasksthatarealreadyscheduledbutnotyetexecutedareneverrun,andnewtaskscannotbescheduled. (Thisproblem,called"threadleakage"isdescribedinSection7.3,alongwithtechniquesforavoidingit.) OutOfTimeinListing6.9illustrateshowaTimercanbecomeconfusedinthismannerand,asconfusionlovescompany, howtheTimersharesitsconfusionwiththenexthaplesscallerthattriestosubmitaTimerTask.Youmightexpectthe program to run for six seconds and exit, but what actually happens is that it terminates after one second with an IllegalStateException whose message text is "Timer already cancelled". ScheduledThreadPoolExecutor deals properlywithillͲbehavedtasks;thereislittlereasontouseTimerinJava5.0orlater. If you need to build yourown scheduling service, you may stillbe able to take advantage of the library by using a DelayQueue, a BlockingQueue implementation that provides the scheduling functionality of ScheduledThreadPoolExecutor.ADelayQueuemanagesacollectionofDelayedobjects.ADelayedhasadelaytime associated with it: DelayQueue lets you take an element only if its delay has expired. Objects are returned from a DelayQueueorderedbythetimeassociatedwiththeirdelay. 6.3.FindingExploitableParallelism TheExecutorframeworkmakesiteasytospecifyanexecutionpolicy,butinordertouseanExecutor,youhavetobe abletodescribeyourtaskasaRunnable.Inmostserverapplications,thereisanobvioustaskboundary:asingleclient request.Butsometimesgoodtaskboundariesarenotquitesoobvious,asinmanydesktopapplications.Theremayalso beexploitableparallelismwithinasingleclientrequestinserverapplications,asissometimesthecaseindatabase servers.(Forafurtherdiscussionofthecompetingdesignforcesinchoosingtaskboundaries,see[CPJ4.4.1.1].) Listing6.9.ClassIllustratingConfusingTimerBehavior. public class OutOfTime { public static void main(String[] args) throws Exception { Timer timer = new Timer(); timer.schedule(new ThrowTask(), 1); SECONDS.sleep(1); timer.schedule(new ThrowTask(), 1); SECONDS.sleep(5); } static class ThrowTask extends TimerTask { public void run() { throw new RuntimeException(); } } } Inthissectionwedevelopseveralversionsofacomponentthatadmitvaryingdegreesofconcurrency.Oursample componentisthepageͲrenderingportionofabrowserapplication,whichtakesapageofHTMLandrendersitintoan imagebuffer.Tokeepitsimple,weassumethattheHTMLconsistsonlyofmarkeduptextinterspersedwithimage elementswithpreͲspecifieddimensionsandURLs. 6.3.1.Example:SequentialPageRenderer ThesimplestapproachistoprocesstheHTMLdocumentsequentially.Astextmarkupisencountered,renderitintothe imagebuffer;asimagereferencesareencountered,fetchtheimageoverthenetworkanddrawitintotheimagebuffer aswell.Thisiseasytoimplementandrequirestouchingeachelementoftheinputonlyonce(itdoesn'tevenrequire buffering the document), but is likely to annoy the user, who may have to wait a long time before all the text is rendered.  795BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution Alessannoyingbutstillsequentialapproachinvolvesrenderingthetextelementsfirst,leavingrectangularplaceholders fortheimages,andaftercompletingtheinitialpassonthedocument,goingbackanddownloadingtheimagesand drawingthemintotheassociatedplaceholder.ThisapproachisshowninSingleThreadRendererinListing6.10. DownloadinganimagemostlyinvolveswaitingforI/Otocomplete,andduringthistimetheCPUdoeslittlework.Sothe sequentialapproachmayunderutilizetheCPU,andalsomakestheuserwaitlongerthannecessarytoseethefinished page.Wecanachievebetterutilizationandresponsivenessbybreakingtheproblemintoindependenttasksthatcan executeconcurrently. Listing6.10.RenderingPageElementsSequentially. public class SingleThreadRenderer { void renderPage(CharSequence source) { renderText(source); List imageData = new ArrayList(); for (ImageInfo imageInfo : scanForImageInfo(source)) imageData.add(imageInfo.downloadImage()); for (ImageData data : imageData) renderImage(data); } } 6.3.2.ResultǦbearingTasks:CallableandFuture TheExecutorframeworkusesRunnableasitsbasictaskrepresentation.Runnableisafairlylimitingabstraction;run cannotreturnavalueorthrowcheckedexceptions,althoughitcanhavesideeffectssuchaswritingtoalogfileor placingaresultinashareddatastructure. ManytasksareeffectivelydeferredcomputationsͲexecutingadatabasequery,fetchingaresourceoverthenetwork, orcomputingacomplicatedfunction.Forthesetypesoftasks,Callableisabetterabstraction:itexpectsthatthemain entrypoint, call,willreturnavalueandanticipatesthatitmightthrowanexception.[7] Executorsincludesseveral utilitymethodsforwrappingothertypesoftasks,includingRunnableandjava.security.PrivilegedAction,witha Callable. [7]ToexpressanonͲvalueͲreturningtaskwithCallable,useCallable. RunnableandCallabledescribeabstractcomputationaltasks.Tasksareusuallyfinite:theyhaveaclearstartingpoint andtheyeventuallyterminate.ThelifecycleofataskexecutedbyanExecutorhasfourphases:created,submitted, started, and completed. Since tasks can take a long time to run, we also want to be able to cancel a task. In the Executorframework,tasksthathavebeensubmittedbutnotyetstartedcanalwaysbecancelled,andtasksthathave startedcansometimesbecancellediftheyareresponsivetointerruption.Cancellingataskthathasalreadycompleted hasnoeffect.(CancellationiscoveredingreaterdetailinChapter7.) Future represents the lifecycle of a task and provides methods to test whether the task has completed or been cancelled, retrieve its result, and cancel the task. Callable and Future are shown in Listing 6.11. Implicit in the specificationofFutureisthattasklifecyclecanonlymoveforwards,notbackwardsͲjustliketheExecutorService lifecycle.Onceataskiscompleted,itstaysinthatstateforever. Thebehaviorofgetvariesdependingonthetaskstate(notyetstarted,running,completed).Itreturnsimmediatelyor throws an Exception if the task has already completed, but if not it blocks until the task completes. If the task completes by throwing an exception, get rethrows it wrapped in an ExecutionException; if it was cancelled, get throwsCancellationException.IfgetthrowsExecutionException,theunderlyingexceptioncanberetrievedwith getCause.  80 JavaConcurrencyInPractice Listing6.11.CallableandFutureInterfaces. public interface Callable { V call() throws Exception; } public interface Future { boolean cancel(boolean mayInterruptIfRunning); boolean isCancelled(); boolean isDone(); V get() throws InterruptedException, ExecutionException, CancellationException; V get(long timeout, TimeUnit unit) throws InterruptedException, ExecutionException, CancellationException, TimeoutException; } ThereareseveralwaystocreateaFuturetodescribeatask.The submitmethodsin ExecutorServiceallreturna Future,sothatyoucansubmitaRunnableoraCallabletoanexecutorandgetbackaFuturethatcanbeusedto retrievetheresultorcancelthetask.YoucanalsoexplicitlyinstantiateaFutureTaskforagivenRunnableorCallable. (BecauseFutureTaskimplementsRunnable,itcanbesubmittedtoanExecutorforexecutionorexecuteddirectlyby callingitsrunmethod.) As of Java 6, ExecutorService implementations can override newTaskFor in AbstractExecutorService to control instantiation of the Future corresponding to a submitted Callable or Runnable. The default implementation just createsanewFutureTask,asshowninListing6.12. Listing6.12.DefaultImplementationofnewTaskForinThreadPoolExecutor. protected RunnableFuture newTaskFor(Callable task) { return new FutureTask(task); } SubmittingaRunnableorCallabletoanExecutorconstitutesasafepublication(seeSection3.5)oftheRunnableor Callablefromthesubmittingthreadtothethreadthatwilleventuallyexecutethetask.Similarly,settingtheresult valueforaFutureconstitutesasafepublicationoftheresultfromthethreadinwhichitwascomputedtoanythread thatretrievesitviaget. 6.3.3.Example:PageRendererwithFuture Asafirststeptowardsmakingthepagerenderermoreconcurrent,let'sdivideitintotwotasks,onethatrendersthe textandonethatdownloadsalltheimages.(BecauseonetaskislargelyCPUͲboundandtheotherislargelyI/OͲbound, thisapproachmayyieldimprovementsevenonsingleͲCPUsystems.) Callable and Future can help us express the interaction between these cooperating tasks. In FutureRenderer in Listing6.13,wecreateaCallabletodownloadalltheimages,andsubmitittoanExecutorService.Thisreturnsa Futuredescribingthetask'sexecution;whenthemaintaskgetstothepointwhereitneedstheimages,itwaitsforthe resultbycallingFuture.get.Ifwe'relucky,theresultswillalreadybereadybythetimeweask;otherwise,atleastwe gotaheadstartondownloadingtheimages. ThestateͲdependentnatureof get meansthat the callerneednotbeawareofthestateofthe task,and thesafe publicationpropertiesoftasksubmissionandresultretrievalmakethisapproachthreadͲsafe.Theexceptionhandling codesurroundingFuture.getdealswithtwopossibleproblems:thatthetaskencounteredanException,orthethread callinggetwasinterruptedbeforetheresultswereavailable.(SeeSections5.5.2and5.4.) FutureRendererallowsthetexttoberenderedconcurrentlywithdownloadingtheimagedata.Whenalltheimages aredownloaded,theyarerenderedontothepage.Thisisanimprovementinthattheuserseesaresultquicklyandit exploitssomeparallelism,butwecandoconsiderablybetter.Thereisnoneedforuserstowaitforalltheimagestobe downloaded;theywouldprobablyprefertoseeindividualimagesdrawnastheybecomeavailable. 6.3.4.LimitationsofParallelizingHeterogeneousTasks Inthelastexample,wetriedtoexecutetwodifferenttypesoftasksinparallelͲdownloadingtheimagesandrendering thepage.Butobtainingsignificantperformanceimprovementsbytryingtoparallelizesequentialheterogeneoustasks canbetricky. Twopeoplecandividetheworkofcleaningthedinnerdishesfairlyeffectively:onepersonwasheswhiletheotherdries. However,assigningadifferenttypeoftasktoeachworkerdoesnotscalewell;ifseveralmorepeopleshowup,itisnot obvioushowtheycanhelpwithoutgettinginthewayorsignificantlyrestructuringthedivisionoflabor.Withoutfinding finerͲgrainedparallelismamongsimilartasks,thisapproachwillyielddiminishingreturns.  815BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution Afurtherproblemwithdividingheterogeneoustasksamongmultipleworkersisthatthetasksmayhavedisparatesizes. IfyoudividetasksAandBbetweentwoworkersbutAtakestentimesaslongasB,you'veonlyspeededupthetotal processby9%.Finally,dividingataskamongmultipleworkersalwaysinvolvessomeamountofcoordinationoverhead; forthedivisiontobeworthwhile,thisoverheadmustbemorethancompensatedbyproductivityimprovementsdueto parallelism. FutureRendererusestwotasks:oneforrenderingtextandonefordownloadingtheimages.Ifrenderingthetextis muchfasterthandownloadingtheimages,asisentirelypossible,theresultingperformanceisnotmuchdifferentfrom thesequentialversion,butthecodeisalotmorecomplicated.Andthebestwecandowithtwothreadsisspeedthings upbyafactoroftwo.Thus,tryingtoincreaseconcurrencybyparallelizingheterogeneousactivitiescanbealotofwork, and there is a limit to how much additional concurrency you can get out of it. (See Sections 11.4.2 and 11.4.3 for anotherexampleofthesamephenomenon.) Listing6.13.WaitingforImageDownloadwithFuture. public class FutureRenderer { private final ExecutorService executor = ...; void renderPage(CharSequence source) { final List imageInfos = scanForImageInfo(source); Callable> task = new Callable>() { public List call() { List result = new ArrayList(); for (ImageInfo imageInfo : imageInfos) result.add(imageInfo.downloadImage()); return result; } }; Future> future = executor.submit(task); renderText(source); try { List imageData = future.get(); for (ImageData data : imageData) renderImage(data); } catch (InterruptedException e) { // Re-assert the thread's interrupted status Thread.currentThread().interrupt(); // We don't need the result, so cancel the task too future.cancel(true); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } }  The real performance payoff of dividing a program's workload into tasks comes when there are a large number of independent,homogeneoustasksthatcanbeprocessedconcurrently. 6.3.5.CompletionService:ExecutorMeetsBlockingQueue IfyouhaveabatchofcomputationstosubmittoanExecutorandyouwanttoretrievetheirresultsastheybecome available,youcouldretaintheFutureassociatedwitheachtaskandrepeatedlypollforcompletionbycallinggetwitha timeoutofzero.Thisispossible,buttedious.Fortunatelythereisabetterway:acompletionservice. CompletionServicecombinesthefunctionalityofanExecutorandaBlockingQueue.YoucansubmitCallabletasks toitforexecutionandusethequeueͲlikemethodstakeandpolltoretrievecompletedresults,packagedasFutures, astheybecomeavailable.ExecutorCompletionServiceimplementsCompletionService,delegatingthecomputation toanExecutor. The implementation of ExecutorCompletionService is quite straightforward. The constructor creates a BlockingQueuetoholdthecompletedresults.Future-Taskhasadonemethodthatiscalledwhenthecomputation completes.Whenataskissubmitted,itiswrappedwithaQueueingFuture,asubclassofFutureTaskthatoverrides donetoplacetheresultontheBlockingQueue,asshowninListing6.14.Thetakeandpollmethodsdelegatetothe BlockingQueue,blockingifresultsarenotyetavailable.  82 JavaConcurrencyInPractice Listing6.14.QueueingFutureClassUsedByExecutorCompletionService. private class QueueingFuture extends FutureTask { QueueingFuture(Callable c) { super(c); } QueueingFuture(Runnable t, V r) { super(t, r); } protected void done() { completionQueue.add(this); } } 6.3.6.Example:PageRendererwithCompletionService WecanuseaCompletionServicetoimprovetheperformanceofthepagerendererintwoways:shortertotalruntime andimprovedresponsiveness.Wecancreateaseparatetaskfordownloadingeachimageandexecutetheminathread pool,turningthesequentialdownloadintoaparallelone:thisreducestheamountoftimetodownloadalltheimages. AndbyfetchingresultsfromtheCompletionServiceandrenderingeachimageassoonasitisavailable,wecangive theuseramoredynamicandresponsiveuserinterface.ThisimplementationisshowninRendererinListing6.15. Listing6.15.UsingCompletionServicetoRenderPageElementsastheyBecomeAvailable. public class Renderer { private final ExecutorService executor; Renderer(ExecutorService executor) { this.executor = executor; } void renderPage(CharSequence source) { final List info = scanForImageInfo(source); CompletionService completionService = new ExecutorCompletionService(executor); for (final ImageInfo imageInfo : info) completionService.submit(new Callable() { public ImageData call() { return imageInfo.downloadImage(); } }); renderText(source); try { for (int t = 0, n = info.size(); t < n; t++) { Future f = completionService.take(); ImageData imageData = f.get(); renderImage(imageData); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); } catch (ExecutionException e) { throw launderThrowable(e.getCause()); } } } Multiple ExecutorCompletionServices can share a single Executor, so it is perfectly sensible to create an ExecutorCompletionServicethatisprivatetoaparticularcomputationwhilesharingacommonExecutor.Whenused inthisway,aCompletionServiceactsasahandleforabatchofcomputationsinmuchthesamewaythataFuture actsasahandleforasinglecomputation.ByrememberinghowmanytasksweresubmittedtotheCompletionService andcountinghowmanycompletedresultsareretrieved,youcanknowwhenalltheresultsforagivenbatchhavebeen retrieved,evenifyouuseasharedExecutor. 6.3.7.PlacingTimeLimitsonTasks Sometimes,ifanactivitydoesnotcompletewithinacertainamountoftime,theresultisnolongerneededandthe activitycanbeabandoned.Forexample,awebapplicationmayfetchitsadvertisementsfromanexternaladserver,but iftheadisnotavailablewithintwoseconds,itinsteaddisplaysadefaultadvertisementsothatadunavailabilitydoes notunderminethesite'sresponsivenessrequirements.Similarly,aportalsitemayfetchdatainparallelfrommultiple datasources,butmaybewillingtowaitonlyacertainamountoftimefordatatobeavailablebeforerenderingthepage withoutit. Theprimarychallengeinexecutingtaskswithinatimebudgetismakingsurethatyoudon'twaitlongerthanthetime budget to get an answer or find out that one is not forthcoming. The timed version of Future.get supports this requirement:itreturnsassoonastheresultisready,butthrowsTimeoutExceptioniftheresultisnotreadywithinthe timeoutperiod.  835BPartII:StructuringConcurrentApplications Ͳ18BChapter6.TaskExecution A secondary problem when using timed tasks is to stop them when they run out of time, so they do not waste computingresourcesbycontinuingtocomputearesultthatwillnotbeused.Thiscanbeaccomplishedbyhavingthe taskstrictlymanageitsowntimebudgetandabortifitrunsoutoftime,orbycancellingthetaskifthetimeoutexpires. Again, Future can help; if a timed get completes with a TimeoutException, you can cancel the task through the Future.Ifthetaskiswrittentobecancellable(seeChapter7),itcanbeterminatedearlysoasnottoconsumeexcessive resources.ThistechniqueisusedinListings6.13and6.16. Listing6.16showsatypicalapplicationofatimedFuture.get.Itgeneratesacompositewebpagethatcontainsthe requestedcontentplusanadvertisementfetchedfromanadserver.ItsubmitstheadͲfetchingtasktoanexecutor, computestherestofthepagecontent,andthenwaitsfortheaduntilitstimebudgetrunsout.[8]Ifthegettimesout,it cancels[9]theadͲfetchingtaskandusesadefaultadvertisementinstead. [8]Thetimeoutpassedtogetiscomputedbysubtractingthecurrenttimefromthedeadline;thismayinfactyieldanegativenumber,butallthe timedmethodsinjava.util.concurrenttreatnegativetimeoutsaszero,sonoextracodeisneededtodealwiththiscase. [9]ThetrueparametertoFuture.cancelmeansthatthetaskthreadcanbeinterruptedifthetaskiscurrentlyrunning;seeChapter7. 6.3.8.Example:ATravelReservationsPortal ThetimeͲbudgetingapproachintheprevioussectioncanbeeasilygeneralizedtoanarbitrarynumberoftasks.Consider atravelreservationportal:theuserenterstraveldatesandrequirementsandtheportalfetchesanddisplaysbidsfrom anumberofairlines,hotelsorcarrentalcompanies.Dependingonthecompany,fetchingabidmightinvolveinvokinga webservice,consultingadatabase,performinganEDItransaction,orsomeothermechanism.Ratherthanhavethe responsetimeforthepagebedrivenbytheslowestresponse,itmaybepreferabletopresentonlytheinformation available within a given time budget. For providers that do not respond in time, the page could either omit them completelyordisplayaplaceholdersuchas"DidnothearfromAirJavaintime." Listing6.16.FetchinganAdvertisementwithaTimeBudget. Page renderPageWithAd() throws InterruptedException { long endNanos = System.nanoTime() + TIME_BUDGET; Future f = exec.submit(new FetchAdTask()); // Render the page while waiting for the ad Page page = renderPageBody(); Ad ad; try { // Only wait for the remaining time budget long timeLeft = endNanos - System.nanoTime(); ad = f.get(timeLeft, NANOSECONDS); } catch (ExecutionException e) { ad = DEFAULT_AD; } catch (TimeoutException e) { ad = DEFAULT_AD; f.cancel(true); } page.setAd(ad); return page; } Fetchingabidfromonecompanyisindependentoffetchingbidsfromanother,sofetchingasinglebidisasensibletask boundarythatallowsbidretrievaltoproceedconcurrently.Itwouldbeeasyenoughtocreatentasks,submitthemtoa threadpool,retaintheFutures,anduseatimedgettofetcheachresultsequentiallyviaitsFuture,butthereisan eveneasierwayͲinvokeAll. Listing 6.17 uses the timed version of invokeAll to submit multiple tasks to an ExecutorService and retrieve the results.TheinvokeAllmethodtakesacollectionoftasksandreturnsacollectionofFutures.Thetwocollectionshave identicalstructures;invokeAlladdstheFuturestothereturnedcollectionintheorderimposedbythetaskcollection's iterator,thusallowingthecallertoassociateaFuturewiththeCallableitrepresents.ThetimedversionofinvokeAll willreturnwhenallthetaskshavecompleted,thecallingthreadisinterrupted,orthetimeoutexpires.Anytasksthat are not complete when the timeout expires are cancelled. On return from invokeAll, each task will have either completednormallyorbeencancelled;theclientcodecancallgetorisCancelledtofindoutwhich. Summary Structuring applications around the execution of tasks can simplify development and facilitate concurrency. The Executor framework permits you to decouple task submission from execution policy and supports a rich variety of executionpolicies;wheneveryoufindyourselfcreatingthreadstoperformtasks,considerusinganExecutorinstead. Tomaximizethebenefitofdecomposinganapplicationintotasks,youmustidentifysensibletaskboundaries.Insome  84 JavaConcurrencyInPractice applications,theobvioustaskboundariesworkwell,whereasinotherssomeanalysismayberequiredtouncoverfinerͲ grainedexploitableparallelism. Listing6.17.RequestingTravelQuotesUnderaTimeBudget. private class QuoteTask implements Callable { private final TravelCompany company; private final TravelInfo travelInfo; ... public TravelQuote call() throws Exception { return company.solicitQuote(travelInfo); } } public List getRankedTravelQuotes( TravelInfo travelInfo, Set companies, Comparator ranking, long time, TimeUnit unit) throws InterruptedException { List tasks = new ArrayList(); for (TravelCompany company : companies) tasks.add(new QuoteTask(company, travelInfo)); List> futures = exec.invokeAll(tasks, time, unit); List quotes = new ArrayList(tasks.size()); Iterator taskIter = tasks.iterator(); for (Future f : futures) { QuoteTask task = taskIter.next(); try { quotes.add(f.get()); } catch (ExecutionException e) { quotes.add(task.getFailureQuote(e.getCause())); } catch (CancellationException e) { quotes.add(task.getTimeoutQuote(e)); } } Collections.sort(quotes, ranking); return quotes; }  855BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown Chapter7.CancellationandShutdown Itiseasytostarttasksandthreads.Mostofthetimeweallowthemtodecidewhentostopbylettingthemrunto completion. Sometimes, however, we want to stop tasks or threads earlier than they would on their own, perhaps becausetheusercancelledanoperationortheapplicationneedstoshutdownquickly. Gettingtasksandthreadstostopsafely,quickly,andreliablyisnotalwayseasy.Javadoesnotprovideanymechanism forsafelyforcingathreadtostopwhatitisdoing.[1]Instead,itprovidesinterruption,acooperativemechanismthatlets onethreadaskanothertostopwhatitisdoing. [1]ThedeprecatedThread.stopandsuspendmethodswereanattempttoprovidesuchamechanism,butwerequicklyrealizedtobeseriously flawed and should be avoided. See http://java.sun.com/j2se/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html for an explanation of the problemswiththesemethods. Thecooperativeapproachisrequiredbecausewerarelywantatask,thread,orservicetostopimmediately,sincethat could leave shared data structures in an inconsistent state. Instead, tasks and services can be coded so that, when requested,theycleanupanyworkcurrentlyinprogressandthenterminate.Thisprovidesgreaterflexibility,sincethe taskcodeitselfisusuallybetterabletoassessthecleanuprequiredthanisthecoderequestingcancellation. EndͲofͲlifecycle issues can complicate the design and implementation of tasks, services, and applications, and this importantelementofprogramdesignistoooftenignored.Dealingwellwithfailure,shutdown,andcancellationisone ofthecharacteristicsthatdistinguishawellͲbehavedapplicationfromonethatmerelyworks.Thischapteraddresses mechanisms for cancellation and interruption, and how to code tasks and services to be responsive to cancellation requests. 7.1.TaskCancellation Anactivityiscancellableifexternalcodecanmoveittocompletionbeforeitsnormalcompletion.Thereareanumber ofreasonswhyyoumightwanttocancelanactivity: UserͲrequestedcancellation.Theuserclickedonthe"cancel"buttoninaGUIapplication,orrequestedcancellation throughamanagementinterfacesuchasJMX(JavaManagementExtensions). TimeͲlimited activities. An application searches a problem space for a finite amount of time and chooses the best solutionfoundwithinthattime.Whenthetimerexpires,anytasksstillsearchingarecancelled. Applicationevents.Anapplicationsearchesaproblemspacebydecomposingitsothatdifferenttaskssearchdifferent regionsoftheproblemspace.Whenonetaskfindsasolution,allothertasksstillsearchingarecancelled. Errors. A web crawler searches for relevant pages, storing pages or summary data to disk. When a crawler task encountersanerror(forexample,thediskisfull),othercrawlingtasksarecancelled,possiblyrecordingtheircurrent statesothattheycanberestartedlater. Shutdown.Whenanapplicationorserviceisshutdown,somethingmustbedoneaboutworkthatiscurrentlybeing processedorqueuedforprocessing.Inagracefulshutdown,taskscurrentlyinprogressmightbeallowedtocomplete; inamoreimmediateshutdown,currentlyexecutingtasksmightbecancelled. ThereisnosafewaytopreemptivelystopathreadinJava,andthereforenosafewaytopreemptivelystopatask.There are only cooperative mechanisms, by which the task and the code requesting cancellation follow an agreedͲupon protocol. Onesuchcooperativemechanismissettinga"cancellationrequested"flagthatthetaskchecksperiodically;ifitfinds the flag set, the task terminates early. PrimeGenerator in Listing 7.1, which enumerates prime numbers until it is cancelled, illustrates this technique. The cancel method sets the cancelled flag, and the main loop polls this flag beforesearchingforthenextprimenumber.(Forthistoworkreliably,cancelledmustbevolatile.) Listing7.2showsasampleuseofthisclassthatletstheprimegeneratorrunforonesecondbeforecancellingit.The generator won't necessarily stop after exactly one second, since there may be some delay between the time that cancellationisrequestedandthetimethattherunloopnextchecksforcancellation.Thecancelmethodiscalledfrom afinallyblocktoensurethattheprimegeneratoriscancelledevenifthecalltosleepisinterrupted.Ifcancelwere notcalled,theprimeͲseekingthreadwouldrunforever,consumingCPUcyclesandpreventingtheJVMfromexiting.  86 JavaConcurrencyInPractice Ataskthatwantstobecancellablemusthaveacancellationpolicythatspecifiesthe"how","when",and"what"of cancellationͲhowothercodecanrequestcancellation,whenthetaskcheckswhethercancellationhasbeenrequested, andwhatactionsthetasktakesinresponsetoacancellationrequest. Consider the realͲworld example of stopping payment on a check. Banks have rules about how to submit a stopͲ paymentrequest,whatresponsivenessguaranteesitmakesinprocessingsuchrequests,andwhatproceduresitfollows whenpaymentisactuallystopped(suchasnotifyingtheotherbankinvolvedinthetransactionandassessingafee againstthe payer’saccount).Takentogether,theseprocedures andguaranteescomprise the cancellationpolicyfor checkpayment. Listing7.1.UsingaVolatileFieldtoHoldCancellationState. @ThreadSafe public class PrimeGenerator implements Runnable { @GuardedBy("this") private final List primes = new ArrayList(); private volatile boolean cancelled; public void run() { BigInteger p = BigInteger.ONE; while (!cancelled ) { p = p.nextProbablePrime(); synchronized (this) { primes.add(p); } } } public void cancel() { cancelled = true; } public synchronized List get() { return new ArrayList(primes); } } Listing7.2.GeneratingaSecond'sWorthofPrimeNumbers. List aSecondOfPrimes() throws InterruptedException { PrimeGenerator generator = new PrimeGenerator(); new Thread(generator).start(); try { SECONDS.sleep(1); } finally { generator.cancel(); } return generator.get(); } PrimeGeneratorusesasimplecancellationpolicy:clientcoderequestscancellationbycallingcancel,PrimeGenerator checksforcancellationonceperprimefoundandexitswhenitdetectscancellationhasbeenrequested. 7.1.1.Interruption ThecancellationmechanisminPrimeGeneratorwilleventuallycausetheprimeͲseekingtasktoexit,butitmighttakea while.If,however,ataskthatusesthisapproachcallsablockingmethodsuchasBlockingQueue.put,wecouldhavea moreseriousproblemͲthetaskmightnevercheckthecancellationflagandthereforemightneverterminate. BrokenPrimeProducerinListing7.3illustratesthisproblem.Theproducerthreadgeneratesprimesandplacesthemon ablockingqueue.Iftheproducergetsaheadoftheconsumer,thequeuewillfillupandputwillblock.Whathappensif theconsumertriestocanceltheproducertaskwhileitisblockedinput?Itcancallcancelwhichwillsetthecancelled flagͲbut the producer will never check the flag because it will never emerge from the blocking put (because the consumerhasstoppedretrievingprimesfromthequeue). AswehintedinChapter5,certainblockinglibrarymethodssupportinterruption.Threadinterruptionisacooperative mechanismforathreadtosignalanotherthreadthatitshould,atitsconvenienceandifitfeelslikeit,stopwhatitis doinganddosomethingelse. ThereisnothingintheAPIorlanguagespecificationthattiesinterruptiontoanyspecificcancellationsemantics,butin practice,usinginterruptionforanythingbutcancellationisfragileanddifficulttosustaininlargerapplications. Eachthreadhasabooleaninterruptedstatus;interruptingathreadsetsitsinterruptedstatustotrue.Threadcontains methods for interrupting a thread and querying the interrupted status of a thread, as shown in Listing 7.4. The  875BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown interruptmethodinterruptsthetargetthread,andisInterruptedreturnstheinterruptedstatusofthetargetthread. The poorly named static interrupted method clears the interrupted status of the current thread and returns its previousvalue;thisistheonlywaytocleartheinterruptedstatus. BlockinglibrarymethodslikeThread.sleepandObject.waittrytodetectwhenathreadhasbeeninterruptedand return early. They respond to interruption by clearing the interrupted status and throwing InterruptedException, indicating that the blocking operation completed early due to interruption. The JVM makes no guarantees on how quicklyablockingmethodwilldetectinterruption,butinpracticethishappensreasonablyquickly. Listing7.3.UnreliableCancellationthatcanLeaveProducersStuckinaBlockingOperation.Don'tDothis. class BrokenPrimeProducer extends Thread { private final BlockingQueue queue; private volatile boolean cancelled = false; BrokenPrimeProducer(BlockingQueue queue) { this.queue = queue; } public void run() { try { BigInteger p = BigInteger.ONE; while (!cancelled) queue.put(p = p.nextProbablePrime()); } catch (InterruptedException consumed) { } } public void cancel() { cancelled = true; } } void consumePrimes() throws InterruptedException { BlockingQueue primes = ...; BrokenPrimeProducer producer = new BrokenPrimeProducer(primes); producer.start(); try { while (needMorePrimes()) consume(primes.take()); } finally { producer.cancel(); } } Listing7.4.InterruptionMethodsinThread. public class Thread { public void interrupt() { ... } public boolean isInterrupted() { ... } public static boolean interrupted() { ... } ... } Ifathreadisinterruptedwhenitisnotblocked,itsinterruptedstatusisset,anditisuptotheactivitybeingcancelledto poll the interrupted status to detect interruption. In this way interruption is "sticky” if it doesn't trigger an InterruptedException,evidenceofinterruptionpersistsuntilsomeonedeliberatelyclearstheinterruptedstatus. Calling interrupt does not necessarily stop the target thread from doing what it is doing; it merely delivers the messagethatinterruptionhasbeenrequested. Agoodwaytothinkaboutinterruptionisthatitdoesnotactuallyinterruptarunningthread;itjustrequeststhatthe threadinterruptitselfatthenextconvenientopportunity.(Theseopportunitiesarecalledcancellationpoints.)Some methods,suchas wait, sleep,and join,takesuchrequestsseriously,throwinganexceptionwhentheyreceivean interruptrequestorencounteranalreadysetinterruptstatusuponentry.Wellbehavedmethodsmaytotallyignore suchrequestssolongastheyleavetheinterruptionrequestinplacesothatcallingcodecandosomethingwithit. Poorlybehavedmethodsswallowtheinterruptrequest,thusdenyingcodefurtherupthecallstacktheopportunityto actonit. Thestaticinterruptedmethodshouldbeusedwithcaution,becauseitclearsthecurrentthread'sinterruptedstatus.If youcallinterruptedanditreturnsTRue,unlessyouareplanningtoswallowtheinterruption,youshoulddosomething  88 JavaConcurrencyInPractice withitͲeitherthrowInterruptedExceptionorrestoretheinterruptedstatusbycallinginterruptagain,asinListing 5.10onpage94. BrokenPrimeProducerillustrateshowcustomcancellationmechanismsdonotalwaysinteractwellwithblockinglibrary methods. If you code your tasks to be responsive to interruption, you can use interruption as your cancellation mechanismandtakeadvantageoftheinterruptionsupportprovidedbymanylibraryclasses. Interruptionisusuallythemostsensiblewaytoimplementcancellation. BrokenPrimeProducercanbeeasilyfixed(andsimplified)byusinginterruptioninsteadofabooleanflagtorequest cancellation,asshowninListing7.5.Therearetwopointsineachloopiterationwhereinterruptionmaybedetected:in theblockingputcall,andbyexplicitlypollingtheinterruptedstatusintheloopheader.Theexplicittestisnotstrictly necessaryherebecauseoftheblockingputcall,butitmakesPrimeProducermoreresponsivetointerruptionbecause itchecksforinterruptionbeforestartingthelengthytaskofsearchingforaprime,ratherthanafter.Whencallsto interruptible blocking methods are not frequent enough to deliver the desired responsiveness, explicitly testing the interruptedstatuscanhelp. Listing7.5.UsingInterruptionforCancellation. class PrimeProducer extends Thread { private final BlockingQueue queue; PrimeProducer(BlockingQueue queue) { this.queue = queue; } public void run() { try { BigInteger p = BigInteger.ONE; while (!Thread.currentThread().isInterrupted()) queue.put(p = p.nextProbablePrime()); } catch (InterruptedException consumed) { /* Allow thread to exit */ } } public void cancel() { interrupt(); } } 7.1.2.InterruptionPolicies Just as tasks should have a cancellation policy, threads should have an interruption policy. An interruption policy determineshowathreadinterpretsaninterruptionrequestͲwhatitdoes(ifanything)whenoneisdetected,whatunits ofworkareconsideredatomicwithrespecttointerruption,andhowquicklyitreactstointerruption. The most sensible interruption policy is some form of threadͲlevel or serviceͲlevel cancellation: exit as quickly as practical,cleaningupifnecessary,andpossiblynotifyingsomeowningentitythatthethreadisexiting.Itispossibleto establish other interruption policies, such as pausing or resuming a service, but threads or thread pools with nonstandardinterruptionpoliciesmayneedtoberestrictedtotasksthathavebeenwrittenwithanawarenessofthe policy. Itisimportanttodistinguishbetweenhowtasksandthreadsshouldreacttointerruption.Asingleinterruptrequest mayhavemorethanonedesiredrecipientinterruptingaworkerthreadinathreadpoolcanmeanboth"cancelthe currenttask"and"shutdowntheworkerthread". Tasksdonotexecuteinthreadstheyown;theyborrowthreadsownedbyaservicesuchasathreadpool.Codethat doesn'townthethread(forathreadpool,anycodeoutsideofthethreadpoolimplementation)shouldbecarefulto preservetheinterruptedstatussothattheowningcodecaneventuallyactonit,evenifthe"guest"codeactsonthe interruptionaswell.(IfyouarehouseͲsittingforsomeone,youdon'tthrowoutthemailthatcomeswhilethey'reawayͲ yousaveitandletthemdealwithitwhentheygetback,evenifyoudoreadtheirmagazines.) ThisiswhymostblockinglibrarymethodssimplythrowInterruptedExceptioninresponsetoaninterrupt.Theywill neverexecuteinathreadtheyown,sotheyimplementthemostreasonablecancellationpolicyfortaskorlibrarycode: getoutofthewayasquicklyaspossibleandcommunicatetheinterruptionbacktothecallersothatcodehigherupon thecallstackcantakefurtheraction. Ataskneedn'tnecessarilydropeverythingwhenitdetectsaninterruptionrequestͲitcanchoosetopostponeituntila moreopportunetimebyrememberingthatitwasinterrupted,finishingthetaskitwasperforming,andthenthrowing  895BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown InterruptedException or otherwise indicating interruption. This technique can protect data structures from corruptionwhenanactivityisinterruptedinthemiddleofanupdate. Ataskshouldnotassumeanythingabouttheinterruptionpolicyofitsexecutingthreadunlessitisexplicitlydesignedto runwithinaservicethathasaspecificinterruptionpolicy.Whetherataskinterpretsinterruptionascancellationor takessomeotheractiononinterruption,itshouldtakecaretopreservetheexecutingthread'sinterruptionstatus.Ifitis not simply going to propagate InterruptedException to its caller, it should restore the interruption status after catchingInterruptedException: Thread.currentThread().interrupt(); Justastaskcodeshouldnotmakeassumptionsaboutwhatinterruptionmeanstoitsexecutingthread,cancellation codeshouldnotmakeassumptionsabouttheinterruptionpolicyofarbitrarythreads.Athreadshouldbeinterrupted only by its owner; the owner can encapsulate knowledge of the thread's interruption policy in an appropriate cancellationmechanismsuchasashutdownmethod. Because each thread has its own interruption policy, you should not interrupt a thread unless you know what interruptionmeanstothatthread. CriticshavederidedtheJavainterruptionfacilitybecauseitdoesnotprovideapreemptiveinterruptioncapabilityand yet forces developers to handle InterruptedException. However, the ability to postpone an interruption request enablesdeveloperstocraftflexibleinterruptionpoliciesthatbalanceresponsivenessandrobustnessasappropriatefor theapplication. 7.1.3.RespondingtoInterruption As mentioned in Section 5.4, when you call an interruptible blocking method such as Thread.sleep or BlockingQueue.put,therearetwopracticalstrategiesforhandlingInterruptedException: x Propagate the exception (possibly after some taskͲspecific cleanup), making your method an interruptible blockingmethod,too;or x Restoretheinterruptionstatussothatcodehigheruponthecallstackcandealwithit. PropagatingInterruptedExceptioncanbeaseasyasaddingInterruptedExceptiontothethrowsclause,asshown bygetNextTaskinListing7.6. Listing7.6.PropagatingInterruptedExceptiontoCallers. BlockingQueue queue; ... public Task getNextTask() throws InterruptedException { return queue.take(); } Ifyoudon'twanttoorcannotpropagateInterruptedException(perhapsbecauseyourtaskisdefinedbyaRunnable), you need to find another way to preserve the interruption request. The standard way to do this is to restore the interrupted status by calling interrupt again. What you should not do is swallow the InterruptedException by catchingitanddoingnothinginthecatchblock,unlessyourcodeisactuallyimplementingtheinterruptionpolicyfora thread.PrimeProducerswallowstheinterrupt,butdoessowiththeknowledgethatthethreadisabouttoterminate andthatthereforethereisnocodehigheruponthecallstackthatneedstoknowabouttheinterruption.Mostcode doesnotknowwhatthreaditwillruninandsoshouldpreservetheinterruptedstatus. Onlycodethatimplementsathread'sinterruptionpolicymayswallowaninterruptionrequest.GeneralͲpurposetask andlibrarycodeshouldneverswallowinterruptionrequests. Activitiesthatdonotsupportcancellationbutstillcallinterruptibleblockingmethodswillhavetocalltheminaloop, retryingwheninterruptionisdetected.Inthiscase,theyshouldsavetheinterruptionstatuslocallyandrestoreitjust beforereturning,asshowninListing7.7,ratherthanimmediatelyuponcatchingInterruptedException.Settingthe interruptedstatustooearlycouldresultinaninfiniteloop,becausemostinterruptibleblocking methodscheck the interruptedstatusonentryandthrowInterruptedExceptionimmediatelyifitisset.(Interruptiblemethodsusually pollforinterruptionbeforeblockingordoinganysignificantwork,soastobeasresponsivetointerruptionaspossible.) Ifyourcodedoesnotcallinterruptibleblockingmethods,itcanstillbemaderesponsivetointerruptionbypollingthe current thread's interrupted status throughout the task code. Choosing a polling frequency is a tradeoff between  90 JavaConcurrencyInPractice efficiencyandresponsiveness.Ifyouhavehighresponsivenessrequirements,youcannotcallpotentiallylongͲrunning methodsthatarenotthemselvesresponsivetointerruption,potentiallyrestrictingyouroptionsforcallinglibrarycode. Cancellationcaninvolvestateotherthantheinterruptionstatus;interruptioncanbeusedtogetthethread'sattention, and information stored elsewhere by the interrupting thread can be used to provide further instructions for the interruptedthread.(Besuretousesynchronizationwhenaccessingthatinformation.) Listing7.7.NonǦcancelableTaskthatRestoresInterruptionBeforeExit. public Task getNextTask(BlockingQueue task = taskExec.submit(r); try { task.get(timeout, unit); } catch (TimeoutException e) { // task will be cancelled below } catch (ExecutionException e) { // exception thrown in task; rethrow throw launderThrowable(e.getCause()); } finally { // Harmless if task already completed task.cancel(true); // interrupt if running } }  When Future.get throws InterruptedException or TimeoutException and you know that the result is no longer neededbytheprogram,cancelthetaskwithFuture.cancel.  7.1.6.DealingwithNonͲinterruptibleBlocking ManyblockinglibrarymethodsrespondtointerruptionbyreturningearlyandthrowingInterruptedException,which makes it easier to build tasks that are responsive to cancellation. However, not all blocking methods or blocking mechanisms are responsive to interruption; if a thread is blocked performing synchronous socket I/O or waiting to acquire an intrinsic lock, interruption has no effect other than setting the thread's interrupted status. We can sometimesconvincethreadsblockedinnonͲinterruptibleactivitiestostopbymeanssimilartointerruption,butthis requiresgreaterawarenessofwhythethreadisblocked. SynchronoussocketI/Oinjava.io.ThecommonformofblockingI/Oinserverapplicationsisreadingorwritingtoa socket. Unfortunately, the read and write methods in InputStream and OutputStream are not responsive to interruption,butclosingtheunderlyingsocketmakesanythreadsblockedinreadorwritethrowaSocketException. Synchronous I/O in java.nio. Interrupting a thread waiting on an InterruptibleChannel causes it to throw ClosedByInterruptException and close the channel (and also causes all other threads blocked on the channel to throw ClosedByInterruptException). Closing an InterruptibleChannel causes threads blocked on channel operationstothrowAsynchronousCloseException.MoststandardChannelsimplementInterruptibleChannel. AsynchronousI/OwithSelector.IfathreadisblockedinSelector.select(injava.nio.channels),wakeupcausesitto returnprematurelybythrowingaClosedSelectorException. Lock acquisition. If a thread is blocked waiting for an intrinsic lock, there is nothing you can do to stop it short of ensuringthatiteventuallyacquiresthelockandmakesenoughprogressthatyoucangetitsattentionsomeotherway. However,theexplicitLockclassesofferthelockInterruptiblymethod,whichallowsyoutowaitforalockandstillbe responsivetointerruptsͲseeChapter13. ReaderThreadinListing7.11showsatechniqueforencapsulatingnonstandardcancellation.ReaderThreadmanagesa singlesocketconnection,readingsynchronouslyfromthesocketandpassinganydatareceivedtoprocessBuffer.To facilitateterminatingauserconnectionorshuttingdowntheserver,ReaderThreadoverridesinterrupttobothdeliver astandardinterruptandclosetheunderlyingsocket;thusinterruptingaReaderThreadmakesitstopwhatitisdoing whetheritisblockedinreadorinaninterruptibleblockingmethod. 7.1.7.EncapsulatingNonstandardCancellationwithNewtaskfor Thetechniqueusedin ReaderThreadtoencapsulatenonstandardcancellationcanberefinedusingthe newTaskFor hookaddedtoThreadPoolExecutorinJava6.WhenaCallableissubmittedtoanExecutorService,submitreturns a Future that can be used to cancel the task. The newTaskFor hook is a factory method that creates the Future representing the task. It returns a RunnableFuture, an interface that extends both Future and Runnable (and is implementedbyFutureTask). CustomizingthetaskFutureallowsyoutooverrideFuture.cancel.Customcancellationcodecanperformloggingor gather statistics on cancellation, and can also be used to cancel activities that are not responsive to interruption.  935BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown ReaderThreadencapsulatescancellationofsocketͲusingthreadsbyoverridinginterrupt;thesamecanbedonefor tasksbyoverridingFuture.cancel. CancellableTask in Listing 7.12 defines a CancellableTask interface that extends Callable and adds a cancel method and a newTask factory method for constructing a RunnableFuture. CancellingExecutor extends ThreadPoolExecutor,andoverridesnewTaskFortoletaCancellableTaskcreateitsownFuture. Listing7.11.EncapsulatingNonstandardCancellationinaThreadbyOverridingInterrupt. public class ReaderThread extends Thread { private final Socket socket; private final InputStream in; public ReaderThread(Socket socket) throws IOException { this.socket = socket; this.in = socket.getInputStream(); } public void interrupt() { try { socket.close(); } catch (IOException ignored) { } finally { super.interrupt(); } } public void run() { try { byte[] buf = new byte[BUFSZ]; while (true) { int count = in.read(buf); if (count < 0) break; else if (count > 0) processBuffer(buf, count); } } catch (IOException e) { /* Allow thread to exit */ } } } SocketUsingTask implements CancellableTask and defines Future.cancel to close the socket as well as call super.cancel.IfaSocketUsingTaskiscancelledthroughitsFuture,thesocketisclosedandtheexecutingthreadis interrupted.Thisincreasesthetask'sresponsivenesstocancellation:notonlycanitsafelycallinterruptibleblocking methodswhileremainingresponsivetocancellation,butitcanalsocallblockingsocketI/Omethods. 7.2.StoppingaThreadǦbasedService Applicationscommonlycreateservicesthatownthreads,suchasthreadpools,andthelifetimeoftheseservicesis usuallylongerthanthatofthemethodthatcreatesthem.Iftheapplicationistoshutdowngracefully,thethreads ownedbytheseservicesneedtobeterminated.Sincethereisnopreemptivewaytostopathread,theymustinstead bepersuadedtoshutdownontheirown. SensibleencapsulationpracticesdictatethatyoushouldnotmanipulateathreadͲinterruptit,modifyitspriority,etc.Ͳ unlessyouownit.ThethreadAPIhasnoformalconceptofthreadownership:athreadisrepresentedwithaThread objectthatcanbefreelysharedlikeanyotherobject.However,itmakessensetothinkofathreadashavinganowner, andthisisusuallytheclassthatcreatedthethread.Soathreadpoolownsitsworkerthreads,andifthosethreadsneed tobeinterrupted,thethreadpoolshouldtakecareofit. Aswithanyotherencapsulatedobject,threadownershipisnottransitive:theapplicationmayowntheserviceandthe servicemayowntheworkerthreads,buttheapplicationdoesn'towntheworkerthreadsandthereforeshouldnot attempttostopthemdirectly.Instead,theserviceshouldprovidelifecyclemethodsforshuttingitselfdownthatalso shut down the owned threads; then the application can shut down the service, and the service can shut down the threads. ExecutorService provides the shutdown and shutdownNow methods; other threadͲowning services should provideasimilarshutdownmechanism. ProvidelifecyclemethodswheneverathreadͲowningservicehasalifetimelongerthanthatofthemethodthatcreated it.   94 JavaConcurrencyInPractice 7.2.1.Example:ALoggingService Mostserverapplicationsuselogging,whichcanbeassimpleasinserting printlnstatementsintothecode.Stream classeslikePrintWriterarethreadͲsafe,sothissimpleapproachwouldrequirenoexplicitsynchronization.[3]However, as we'll see in Section 11.6, inline logging can have some performance costs in highvolume applications. Another alternativeishavethelogcallqueuethelogmessageforprocessingbyanotherthread. [3]Ifyouareloggingmultiplelinesaspartofasinglelogmessage,youmayneedtouseadditionalclientͲsidelockingtopreventundesirable interleavingofoutputfrommultiplethreads.Iftwothreadsloggedmultilinestacktracestothesamestreamwithoneprintlncallperline,the resultswouldbeinterleavedunpredictably,andcouldeasilylooklikeonelargebutmeaninglessstacktrace. Listing7.12.EncapsulatingNonstandardCancellationinaTaskwithNewtaskfor. public interface CancellableTask extends Callable { void cancel(); RunnableFuture newTask(); } @ThreadSafe public class CancellingExecutor extends ThreadPoolExecutor { ... protected RunnableFuture newTaskFor(Callable callable) { if (callable instanceof CancellableTask) return ((CancellableTask) callable).newTask(); else return super.newTaskFor(callable); } } public abstract class SocketUsingTask implements CancellableTask { @GuardedBy("this") private Socket socket; protected synchronized void setSocket(Socket s) { socket = s; } public synchronized void cancel() { try { if (socket != null) socket.close(); } catch (IOException ignored) { } } public RunnableFuture newTask() { return new FutureTask(this) { public boolean cancel(boolean mayInterruptIfRunning) { try { SocketUsingTask.this.cancel(); } finally { return super.cancel(mayInterruptIfRunning); } } }; } } LogWriterinListing7.13showsasimpleloggingserviceinwhichtheloggingactivityismovedtoaseparatelogger thread.Insteadofhavingthethreadthatproducesthemessagewriteitdirectlytotheoutputstream,LogWriterhands itofftotheloggerthreadviaaBlockingQueueandtheloggerthreadwritesitout.ThisisamultipleͲproducer,singleͲ consumerdesign:anyactivitycallinglogisactingasaproducer,andthebackgroundloggerthreadistheconsumer.If theloggerthreadfallsbehind,theBlockingQueueeventuallyblockstheproducersuntiltheloggerthreadcatchesup.  955BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown Listing7.13.ProducerǦConsumerLoggingServicewithNoShutdownSupport. public class LogWriter { private final BlockingQueue queue; private final LoggerThread logger; public LogWriter(Writer writer) { this.queue = new LinkedBlockingQueue(CAPACITY); this.logger = new LoggerThread(writer); } public void start() { logger.start(); } public void log(String msg) throws InterruptedException { queue.put(msg); } private class LoggerThread extends Thread { private final PrintWriter writer; ... public void run() { try { while (true) writer.println(queue.take()); } catch(InterruptedException ignored) { } finally { writer.close(); } } } } ForaservicelikeLogWritertobeusefulinproduction,weneedawaytoterminatetheloggerthreadsoitdoesnot preventtheJVMfromshuttingdownnormally.Stoppingtheloggerthreadiseasyenough,sinceitrepeatedlycallstake, whichisresponsivetointerruption;iftheloggerthreadismodifiedtoexitoncatchingInterruptedException,then interruptingtheloggerthreadstopstheservice. However,simplymakingtheloggerthreadexitisnotaverysatifyingshutdownmechanism.Suchanabruptshutdown discardslogmessagesthatmightbewaitingtobewrittentothelog,but,moreimportantly,threadsblockedin log becausethequeueisfullwillneverbecomeunblocked.Cancellingaproducerconsumeractivityrequirescancellingboth theproducersandtheconsumers.Interruptingtheloggerthreaddealswiththeconsumer,butbecausetheproducersin thiscasearenotdedicatedthreads,cancellingthemisharder. Another approach to shutting down LogWriter would be to set a "shutdown requested" flag to prevent further messagesfrombeingsubmitted,asshowninListing7.14.Theconsumercouldthendrainthequeueuponbeingnotified thatshutdownhasbeenrequested,writingoutanypendingmessagesandunblockinganyproducersblockedinlog. However, this approach has race conditions that make it unreliable. The implementation of log is a checkͲthenͲact sequence:producerscouldobservethattheservicehasnotyetbeenshutdownbutstillqueuemessagesafterthe shutdown,againwiththeriskthattheproducermightgetblockedinlogandneverbecomeunblocked.Therearetricks thatreducethelikelihoodofthis(likehavingtheconsumerwaitseveralsecondsbeforedeclaringthequeuedrained), butthesedonotchangethefundamentalproblem,merelythelikelihoodthatitwillcauseafailure. Listing7.14.UnreliableWaytoAddShutdownSupporttotheLoggingService. public void log(String msg) throws InterruptedException { if (!shutdownRequested) queue.put(msg); else throw new IllegalStateException("logger is shut down"); } ThewaytoprovidereliableshutdownforLogWriteristofixtheracecondition,whichmeansmakingthesubmissionof anewlogmessageatomic.Butwedon'twanttoholdalockwhiletryingtoenqueuethemessage,sinceputcouldblock. Instead,wecanatomicallycheckforshutdownandconditionallyincrementacounterto"reserve"therighttosubmita message,asshowninLogServiceinListing7.15.  96 JavaConcurrencyInPractice 7.2.2.ExecutorServiceShutdown InSection6.2.4,wesawthatExecutorServiceofferstwowaystoshutdown:gracefulshutdownwithshutdown,and abruptshutdownwith shutdownNow.Inanabruptshutdown, shutdownNowreturnsthelistoftasksthathadnotyet startedafterattemptingtocancelallactivelyexecutingtasks. Listing7.15.AddingReliableCancellationtoLogWriter. public class LogService { private final BlockingQueue queue; private final LoggerThread loggerThread; private final PrintWriter writer; @GuardedBy("this") private boolean isShutdown; @GuardedBy("this") private int reservations; public void start() { loggerThread.start(); } public void stop() { synchronized (this) { isShutdown = true; } loggerThread.interrupt(); } public void log(String msg) throws InterruptedException { synchronized (this) { if (isShutdown) throw new IllegalStateException(...); ++reservations; } queue.put(msg); } private class LoggerThread extends Thread { public void run() { try { while (true) { try { synchronized (this) { if (isShutdown && reservations == 0) break; } String msg = queue.take(); synchronized (this) { --reservations; } writer.println(msg); } catch (InterruptedException e) { /* retry */ } } } finally { writer.close(); } } } } Thetwodifferentterminationoptionsofferatradeoffbetweensafetyandresponsiveness:abruptterminationisfaster butriskierbecausetasksmaybeinterruptedinthemiddleofexecution,andnormalterminationisslowerbutsafer becausetheExecutorServicedoesnotshutdownuntilallqueuedtasksareprocessed.OtherthreadͲowningservices shouldconsiderprovidingasimilarchoiceofshutdownmodes. Simple programs can get away with starting and shutting down a global ExecutorService from main. More sophisticatedprogramsarelikelytoencapsulatean ExecutorServicebehindahigherͲlevelservicethatprovidesits ownlifecyclemethods,suchasthevariantofLogServiceinListing7.16thatdelegatestoanExecutorServiceinstead of managing its own threads. Encapsulating an ExecutorService extends the ownership chain from application to servicetothreadbyaddinganotherlink;eachmemberofthechainmanagesthelifecycleoftheservicesorthreadsit owns.  975BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown Listing7.16.LoggingServicethatUsesanExecutorService. public class LogService { private final ExecutorService exec = newSingleThreadExecutor(); ... public void start() { } public void stop() throws InterruptedException { try { exec.shutdown(); exec.awaitTermination(TIMEOUT, UNIT); } finally { writer.close(); } } public void log(String msg) { try { exec.execute(new WriteTask(msg)); } catch (RejectedExecutionException ignored) { } } } 7.2.3.PoisonPills AnotherwaytoconvinceaproducerͲconsumerservicetoshutdowniswithapoisonpill:arecognizableobjectplaced onthequeuethatmeans"whenyougetthis,stop."WithaFIFOqueue,poisonpillsensurethatconsumersfinishthe workontheirqueuebeforeshuttingdown,sinceanyworksubmittedpriortosubmittingthepoisonpillwillberetrieved beforethepill;producersshouldnotsubmitanyworkafterputtingapoisonpillonthequeue.IndexingServicein Listings 7.17, 7.18, and 7.19 shows a singleͲproducer, singleͲconsumer version of the desktop search example from Listing5.8onpage91thatusesapoisonpilltoshutdowntheservice. Listing7.17.ShutdownwithPoisonPill. public class IndexingService { private static final File POISON = new File(""); private final IndexerThread consumer = new IndexerThread(); private final CrawlerThread producer = new CrawlerThread(); private final BlockingQueue queue; private final FileFilter fileFilter; private final File root; class CrawlerThread extends Thread { /* Listing 7.18 */ } class IndexerThread extends Thread { /* Listing 7.19 */ } public void start() { producer.start(); consumer.start(); } public void stop() { producer.interrupt(); } public void awaitTermination() throws InterruptedException { consumer.join(); } } Poisonpillsworkonlywhenthenumberofproducersandconsumersisknown.TheapproachinIndexingServicecan beextendedtomultipleproducersbyhavingeachproducerplaceapillonthequeueandhavingtheconsumerstoponly whenitreceivesNproducerspills.ItcanbeextendedtomultipleconsumersbyhavingeachproducerplaceNconsumerspillson thequeue,thoughthiscangetunwieldywithlargenumbersofproducersandconsumers.Poisonpillsworkreliablyonly withunboundedqueues. 7.2.4.Example:AOneǦshotExecutionService Ifamethodneedstoprocessabatchoftasksanddoesnotreturnuntilallthetasksarefinished,itcansimplifyservice lifecyclemanagementby usingaprivate Executor whoselifetimeisboundedbythat method.(The invokeAlland invokeAnymethodscanoftenbeusefulinsuchsituations.) ThecheckMailmethodinListing7.20checksfornewmailinparallelonanumberofhosts.Itcreatesaprivateexecutor andsubmitsataskforeachhost:itthenshutsdowntheexecutorandwaitsfortermination,whichoccurswhenallthe mailͲcheckingtaskshavecompleted.[4] [4]ThereasonanAtomicBooleanisusedinsteadofavolatile booleanisthatinordertoaccessthehasNewMailflagfromtheinner Runnable,itwouldhavetobefinal,whichwouldprecludemodifyingit.  98 JavaConcurrencyInPractice Listing7.18.ProducerThreadforIndexingService. public class CrawlerThread extends Thread { public void run() { try { crawl(root); } catch (InterruptedException e) { /* fall through */ } finally { while (true) { try { queue.put(POISON); break; } catch (InterruptedException e1) { /* retry */ } } } } private void crawl(File root) throws InterruptedException { ... } } Listing7.19.ConsumerThreadforIndexingService. public class IndexerThread extends Thread { public void run() { try { while (true) { File file = queue.take(); if (file == POISON) break; else indexFile(file); } } catch (InterruptedException consumed) { } } } Listing7.20.UsingaPrivateExecutorWhoseLifetimeisBoundedbyaMethodCall. boolean checkMail(Set hosts, long timeout, TimeUnit unit) throws InterruptedException { ExecutorService exec = Executors.newCachedThreadPool(); final AtomicBoolean hasNewMail = new AtomicBoolean(false); try { for (final String host : hosts) exec.execute(new Runnable() { public void run() { if (checkMail(host)) hasNewMail.set(true); } }); } finally { exec.shutdown(); exec.awaitTermination(timeout, unit); } return hasNewMail.get(); } 7.2.5.LimitationsofShutdownnow When an ExecutorService is shut down abruptly with shutdownNow, it attempts to cancel the tasks currently in progressandreturnsalistoftasksthatweresubmittedbutneverstartedsothattheycanbeloggedorsavedforlater processing.[5] [5]TheRunnableobjectsreturnedbyshutdownNowmightnotbethesameobjectsthatweresubmittedtotheExecutorService:they mightbewrappedinstancesofthesubmittedtasks. However,thereisnogeneralwaytofindoutwhichtasksstartedbutdidnotcomplete.Thismeansthatthereisnoway of knowing the state of the tasks in progress at shutdown time unless the tasks themselves perform some sort of checkpointing.Toknowwhichtaskshavenotcompleted,youneedtoknownotonlywhichtasksdidn'tstart,butalso whichtaskswereinprogresswhentheexecutorwasshutdown.[6] [6]Unfortunately,thereisnoshutdownoptioninwhichtasksnotyetstartedarereturnedtothecallerbuttasksinprogressareallowedto complete;suchanoptionwouldeliminatethisuncertainintermediatestate. TRackingExecutorinListing7.21showsatechniquefordeterminingwhichtaskswereinprogressatshutdowntime.By encapsulatinganExecutorServiceandinstrumentingexecute(andsimilarlysubmit,notshown)torememberwhich tasks were cancelled after shutdown, trackingExecutor can identify which tasks started but did not complete normally. After the executor terminates, getCancelledTasks returns the list of cancelled tasks. In order for this  995BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown techniquetowork,thetasksmustpreservethethread'sinterruptedstatuswhentheyreturn,whichwellbehavedtasks willdoanyway. Listing7.21.ExecutorServicethatKeepsTrackofCancelledTasksAfterShutdown. public class TrackingExecutor extends AbstractExecutorService { private final ExecutorService exec; private final Set tasksCancelledAtShutdown = Collections.synchronizedSet(new HashSet()); ... public List getCancelledTasks() { if (!exec.isTerminated()) throw new IllegalStateException(...); return new ArrayList(tasksCancelledAtShutdown); } public void execute(final Runnable runnable) { exec.execute(new Runnable() { public void run() { try { runnable.run(); } finally { if (isShutdown() && Thread.currentThread().isInterrupted()) tasksCancelledAtShutdown.add(runnable); } } }); } // delegate other ExecutorService methods to exec } WebCrawlerinListing7.22showsanapplicationoftrackingExecutor.Theworkofawebcrawlerisoftenunbounded, soifacrawlermustbeshutdownwemightwanttosaveitsstatesoitcanberestartedlater.CrawlTaskprovidesa getPagemethodthatidentifieswhatpageitisworkingon.Whenthecrawlerisshutdown,boththetasksthatdidnot startandthosethatwerecancelledarescannedandtheirURLsrecorded,sothatpageͲcrawlingtasksforthoseURLscan beaddedtothequeuewhenthecrawlerrestarts. TRackingExecutorhasanunavoidableraceconditionthatcouldmakeityieldfalsepositives:tasksthatareidentifiedas cancelled but actually completed. This arises because the thread pool could be shut down between when the last instructionofthetaskexecutesandwhenthepoolrecordsthetaskascomplete.Thisisnotaproblemiftasksare idempotent(ifperformingthemtwicehasthesameeffectasperformingthemonce),astheytypicallyareinaweb crawler.Otherwise,theapplicationretrievingthecancelledtasksmustbeawareofthisriskandbepreparedtodeal withfalsepositives.  100 JavaConcurrencyInPractice Listing7.22.UsingTRackingExecutorServicetoSaveUnfinishedTasksforLaterExecution. public abstract class WebCrawler { private volatile TrackingExecutor exec; @GuardedBy("this") private final Set urlsToCrawl = new HashSet(); ... public synchronized void start() { exec = new TrackingExecutor( Executors.newCachedThreadPool()); for (URL url : urlsToCrawl) submitCrawlTask(url); urlsToCrawl.clear(); } public synchronized void stop() throws InterruptedException { try { saveUncrawled(exec.shutdownNow()); if (exec.awaitTermination(TIMEOUT, UNIT)) saveUncrawled(exec.getCancelledTasks()); } finally { exec = null; } } protected abstract List processPage(URL url); private void saveUncrawled(List uncrawled) { for (Runnable task : uncrawled) urlsToCrawl.add(((CrawlTask) task).getPage()); } private void submitCrawlTask(URL u) { exec.execute(new CrawlTask(u)); } private class CrawlTask implements Runnable { private final URL url; ... public void run() { for (URL link : processPage(url)) { if (Thread.currentThread().isInterrupted()) return; submitCrawlTask(link); } } public URL getPage() { return url; } } } 7.3.HandlingAbnormalThreadTermination ItisobviouswhenasingleͲthreadedconsoleapplicationterminatesduetoanuncaughtexceptiontheprogramstops runningandproducesastacktracethatisverydifferentfromtypicalprogramoutput.Failureofathreadinaconcurrent applicationisnotalwayssoobvious.Thestacktracemaybeprintedontheconsole,butnoonemaybewatchingthe console.Also,whenathreadfails,theapplicationmayappeartocontinuetowork,soitsfailurecouldgounnoticed. Fortunately,therearemeansofbothdetectingandpreventingthreadsfrom"leaking"fromanapplication. TheleadingcauseofprematurethreaddeathisRuntimeException.Becausetheseexceptionsindicateaprogramming errororotherunrecoverableproblem,theyaregenerallynotcaught.Insteadtheypropagateallthewayupthestack,at whichpointthedefaultbehavioristoprintastacktraceontheconsoleandletthethreadterminate. Theconsequencesofabnormalthreaddeathrangefrombenigntodisastrous,dependingonthethread'sroleinthe application.Losingathreadfromathreadpoolcanhaveperformanceconsequences,butanapplicationthatrunswell witha50Ͳthreadpoolwillprobablyrunfinewitha49Ͳthreadpooltoo.ButlosingtheeventdispatchthreadinaGUI application would be quite noticeableͲthe application would stop processing events and the GUI would freeze. OutOfTime on 124 showed a serious consequence of thread leakage: the service represented by the Timer is permanentlyoutofcommission. JustaboutanycodecanthrowaRuntimeException.Wheneveryoucallanothermethod,youaretakingaleapoffaith thatitwillreturnnormallyorthrowoneofthecheckedexceptionsitssignaturedeclares.Thelessfamiliaryouarewith thecodebeingcalled,themoreskepticalyoushouldbeaboutitsbehavior. TaskͲprocessingthreadssuchastheworkerthreadsinathreadpoolortheSwingeventdispatchthreadspendtheir whole life calling unknown code through an abstraction barrier like Runnable, and these threads should be very skepticalthatthecodetheycallwillbewellbehaved.ItwouldbeverybadifaserviceliketheSwingeventthreadfailed justbecausesomepoorlywritteneventhandlerthrewaNullPointerException.Accordingly,thesefacilitiesshouldcall taskswithinatry-catchblockthatcatchesuncheckedexceptions,orwithinatry-finallyblocktoensurethatifthe  1015BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown threadexitsabnormallytheframeworkisinformedofthisandcantakecorrectiveaction.Thisisoneofthefewtimes whenyoumightwanttoconsidercatchingRuntimeExceptionwhenyouarecallingunknown,untrustedcodethrough anabstractionsuchasRunnable.[7] [7]Thereissomecontroversyoverthesafetyofthistechnique;whenathreadthrowsanuncheckedexception,theentireapplicationmaypossibly becompromised.ButthealternativeͲshuttingdowntheentireapplicationͲisusuallynotpractical. Listing7.23illustratesawaytostructureaworkerthreadwithinathreadpool.Ifataskthrowsanuncheckedexception, itallowsthethreadtodie,butnotbeforenotifyingtheframeworkthatthethreadhasdied.Theframeworkmaythen replacetheworkerthreadwithanewthread,ormaychoosenottobecausethethreadpoolisbeingshutdownorthere arealreadyenoughworkerthreadstomeetcurrentdemand. ThreadPoolExecutorandSwingusethistechniqueto ensurethatapoorlybehavedtaskdoesn'tpreventsubsequenttasksfromexecuting.Ifyouarewritingaworkerthread classthatexecutessubmittedtasks,orcallinguntrustedexternalcode(suchasdynamicallyloadedplugins),useoneof theseapproachestopreventapoorlywrittentaskorpluginfromtakingdownthethreadthathappenstocallit. Listing7.23.TypicalThreadǦpoolWorkerThreadStructure. public void run() { Throwable thrown = null; try { while (!isInterrupted()) runTask(getTaskFromWorkQueue()); } catch (Throwable e) { thrown = e; } finally { threadExited(this, thrown); } } 7.3.1.UncaughtExceptionHandlers The previous section offered a proactive approach to the problem of unchecked exceptions. The Thread API also provides the UncaughtExceptionHandler facility, which lets you detect when a thread dies due to an uncaught exception. The two approaches are complementary: taken together, they provide defenseͲindepth against thread leakage. When a thread exits due to an uncaught exception, the JVM reports this event to an applicationͲprovided UncaughtExceptionHandler(seeListing7.24);ifnohandlerexists,thedefaultbehavioristoprintthestacktraceto System.err.[8] [8]BeforeJava5.0,theonlywaytocontroltheUncaughtExceptionHandlerwasbysubclassingThreadGroup.InJava5.0andlater,you cansetanUncaughtExceptionHandleronaperͲthreadbasiswithThread.setUncaughtExceptionHandler,andcanalsosetthe default UncaughtExceptionHandler with Thread.setDefaultUncaughtExceptionHandler. However, only one of these handlers is calledfirst the JVM looks for a perͲthread handler, then for a ThreadGroup handler. The default handler implementation in ThreadGroupdelegatestoitsparentthreadgroup,andsoonupthechainuntiloneoftheThreadGrouphandlersdealswiththeuncaught exceptionoritbubblesuptothetoplevelthreadgroup.ThetopͲlevelthreadgrouphandlerdelegatestothedefaultsystemhandler(ifoneexists; thedefaultisnone)andotherwiseprintsthestacktracetotheconsole. Listing7.24.UncaughtExceptionHandlerInterface. public interface UncaughtExceptionHandler { void uncaughtException(Thread t, Throwable e); } WhatthehandlershoulddowithanuncaughtexceptiondependsonyourqualityͲofͲservicerequirements.Themost commonresponseistowriteanerrormessageandstacktracetotheapplicationlog,asshowninListing7.25.Handlers canalsotakemoredirectaction,suchastryingtorestartthethread,shuttingdowntheapplication,paginganoperator, orothercorrectiveordiagnosticaction. Listing7.25.UncaughtExceptionHandlerthatLogstheException. public class UEHLogger implements Thread.UncaughtExceptionHandler { public void uncaughtException(Thread t, Throwable e) { Logger logger = Logger.getAnonymousLogger(); logger.log(Level.SEVERE, "Thread terminated with exception: " + t.getName(), e); } }  InlongͲrunningapplications,alwaysuseuncaughtexceptionhandlersforallthreadsthatatleastlogtheexception.  102 JavaConcurrencyInPractice To set an UncaughtExceptionHandler for pool threads, provide a ThreadFactory to the ThreadPoolExecutor constructor.(Aswithallthreadmanipulation,onlythethread'sownershouldchangeitsUncaughtExceptionHandler.) Thestandardthreadpoolsallowanuncaughttaskexceptiontoterminatethepoolthread,butuseatry-finallyblock tobenotifiedwhenthishappenssothethreadcanbereplaced.Withoutanuncaughtexceptionhandlerorotherfailure notificationmechanism,taskscanappeartofailsilently,whichcanbeveryconfusing.Ifyouwanttobenotifiedwhena task fails due to an exception so that you can take some taskͲspecific recovery action, either wrap the task with a RunnableorCallablethatcatchestheexceptionoroverridetheafterExecutehookinThreadPoolExecutor. Somewhat confusingly, exceptions thrown from tasks make it to the uncaught exception handler only for tasks submittedwithexecute;fortaskssubmittedwithsubmit,anythrownexception,checkedornot,isconsideredtobe part of the task's return status. If a task submitted with submit terminates with an exception, it is rethrown by Future.get,wrappedinanExecutionException. 7.4.JVMShutdown TheJVMcanshutdownineitheranorderlyorabruptmanner.Anorderlyshutdownisinitiatedwhenthelast"normal" (nonͲdaemon)threadterminates,someonecallsSystem.exit,orbyotherplatformͲspecificmeans(suchassendinga SIGINTorhittingCtrl-C).WhilethisisthestandardandpreferredwayfortheJVMtoshutdown,itcanalsobeshut downabruptlybycallingRuntime.haltorbykillingtheJVMprocessthroughtheoperatingsystem(suchassendinga SIGKILL). 7.4.1.ShutdownHooks Inanorderlyshutdown,theJVMfirststartsallregisteredshutdownhooks.Shutdownhooksareunstartedthreadsthat areregisteredwithRuntime.addShutdownHook.TheJVMmakesnoguaranteesontheorderinwhichshutdownhooks arestarted.Ifanyapplicationthreads(daemonornondaemon)arestillrunningatshutdowntime,theycontinuetorun concurrently with the shutdown process. When all shutdown hooks have completed, the JVM may choose to run finalizers if runFinalizersOnExit is true, and then halts. The JVM makes no attempt to stop or interrupt any applicationthreadsthatarestillrunningatshutdowntime;theyareabruptlyterminatedwhentheJVMeventuallyhalts. Iftheshutdownhooksorfinalizersdon'tcomplete,thentheorderlyshutdownprocess"hangs"andtheJVMmustbe shutdownabruptly.Inanabruptshutdown,theJVMisnotrequiredtodoanythingotherthanhalttheJVM;shutdown hookswillnotrun. Shutdown hooks should be threadͲsafe: they must use synchronization when accessing shared data and should be carefultoavoiddeadlock,justlikeanyotherconcurrentcode.Further,theyshouldnotmakeassumptionsaboutthe stateoftheapplication(suchaswhetherotherserviceshaveshutdownalreadyorallnormalthreadshavecompleted) oraboutwhytheJVMisshuttingdown,andmustthereforebecodedextremelydefensively.Finally,theyshouldexitas quicklyaspossible,sincetheirexistencedelaysJVMterminationatatimewhentheusermaybeexpectingtheJVMto terminatequickly. Shutdown hooks can be used for service or application cleanup, such as deleting temporary files or cleaning up resourcesthatarenotautomaticallycleanedupbytheOS.Listing7.26showshowLogServiceinListing7.16could registerashutdownhookfromitsstartmethodtoensurethelogfileisclosedonexit. Becauseshutdownhooksallrunconcurrently,closingthelogfilecouldcausetroubleforothershutdownhookswho wanttousethelogger.Toavoidthisproblem,shutdownhooksshouldnotrelyonservicesthatcanbeshutdownbythe applicationorothershutdownhooks.Onewaytoaccomplishthisistouseasingleshutdownhookforallservices, ratherthan oneforeachservice,andhaveitcall aseriesofshutdownactions.Thisensuresthatshutdownactions executesequentiallyinasinglethread,thusavoidingthepossibilityofraceconditionsordeadlockbetweenshutdown actions.Thistechniquecanbeusedwhetherornotyouuseshutdownhooks;executingshutdownactionssequentially ratherthanconcurrentlyeliminatesmanypotentialsourcesoffailure.Inapplicationsthatmaintainexplicitdependency informationamongservices,thistechniquecanalsoensurethatshutdownactionsareperformedintherightorder.  1035BPartII:StructuringConcurrentApplications Ͳ 19BChapter7.CancellationandShutdown Listing7.26.RegisteringaShutdownHooktoStoptheLoggingService. public void start() { Runtime.getRuntime().addShutdownHook(new Thread() { public void run() { try { LogService.this.stop(); } catch (InterruptedException ignored) {} } }); } 7.4.2.DaemonThreads Sometimesyouwanttocreateathreadthatperformssomehelperfunctionbutyoudon'twanttheexistenceofthis threadtopreventtheJVMfromshuttingdown.Thisiswhatdaemonthreadsarefor. Threadsaredividedintotwotypes:normalthreadsanddaemonthreads.WhentheJVMstartsup,allthethreadsit creates(suchasgarbagecollectorandotherhousekeepingthreads)aredaemonthreads,exceptthemainthread.When anewthreadiscreated,itinheritsthedaemonstatusofthethreadthatcreatedit,sobydefaultanythreadscreatedby themainthreadarealsonormalthreads. Normal threads and daemon threads differ only in what happens when they exit. When a thread exits, the JVM performsaninventoryofrunningthreads,andiftheonlythreadsthatareleftaredaemonthreads,itinitiatesanorderly shutdown. Whenthe JVMhalts,anyremainingdaemonthreadsareabandonedͲ finally blocksarenotexecuted, stacksarenotunwoundͲtheJVMjustexits. Daemon threads should be used sparinglyͲfew processing activities can be safely abandoned at any time with no cleanup.Inparticular,itisdangeroustousedaemonthreadsfortasksthatmightperformanysortofI/O.Daemon threadsarebestsavedfor"housekeeping"tasks,suchasabackgroundthreadthatperiodicallyremovesexpiredentries fromaninͲmemorycache. Daemonthreadsarenotagoodsubstituteforproperlymanagingthelifecycleofserviceswithinanapplication. 7.4.3.Finalizers Thegarbagecollectordoesagoodjobofreclaimingmemoryresourceswhentheyarenolongerneeded,butsome resources,suchasfileorsockethandles,mustbeexplicitlyreturnedtotheoperatingsystemwhennolongerneeded.To assist in this, the garbage collector treats objects that have a nontrivial finalize method specially: after they are reclaimedbythecollector,finalizeiscalledsothatpersistentresourcescanbereleased. SincefinalizerscanruninathreadmanagedbytheJVM,anystateaccessedbyafinalizerwillbeaccessedbymorethan onethreadandthereforemustbeaccessedwithsynchronization.Finalizersoffernoguaranteesonwhenorevenifthey run, and they impose a significant performance cost on objects with nontrivial finalizers. They are also extremely difficulttowritecorrectly.[9]Inmostcases,thecombinationoffinallyblocksandexplicitclosemethodsdoesabetter jobofresourcemanagementthanfinalizers;thesoleexceptioniswhenyouneedtomanageobjectsthatholdresources acquiredbynativemethods.Forthesereasonsandothers,workhardtoavoidwritingorusingclasseswithfinalizers (otherthantheplatformlibraryclasses)[EJItem6]. [9]See(Boehm,2005)forsomeofthechallengesinvolvedinwritingfinalizers. Avoidfinalizers. Summary EndͲofͲlifecycle issues for tasks, threads, services, and applications can add complexity to their design and implementation. Java does not provide a preemptive mechanism for cancelling activities or terminating threads. Instead,itprovidesacooperativeinterruptionmechanismthatcanbeusedtofacilitatecancellation,butitisuptoyou to construct protocols for cancellation and use them consistently. Using FutureTask and the Executor framework simplifiesbuildingcancellabletasksandservices.  104 JavaConcurrencyInPractice Chapter8.ApplyingThreadPools Chapter6introducedthetaskexecutionframework,whichsimplifiesmanagementoftaskandthreadlifecyclesand providesasimpleandflexiblemeansfordecouplingtasksubmissionfromexecutionpolicy.Chapter7coveredsomeof the messy details of service lifecycle that arise from using the task execution framework in real applications. This chapterlooksatadvancedoptionsforconfiguringandtuningthreadpools,describeshazardstowatchforwhenusing thetaskexecutionframework,andofferssomemoreadvanced 8.1.ImplicitCouplingsBetweenTasksandExecutionPolicies WeclaimedearlierthattheExecutorframeworkdecouplestasksubmissionfromtaskexecution.Likemanyattemptsat decouplingcomplexprocesses,thiswasabitofanoverstatement.WhiletheExecutorframeworkofferssubstantial flexibilityinspecifyingandmodifyingexecutionpolicies,notalltasksarecompatiblewithallexecutionpolicies.Typesof tasksthatrequirespecificexecutionpoliciesinclude: Dependenttasks.Themostwellbehavedtasksareindependent:thosethatdonotdependonthetiming,results,orside effects of other tasks. When executing independent tasks in a thread pool, you can freely vary the pool size and configurationwithoutaffectinganythingbutperformance.Ontheotherhand,whenyousubmittasksthatdependon othertaskstoathreadpool,youimplicitlycreateconstraintsontheexecutionpolicythatmustbecarefullymanagedto avoidlivenessproblems(seeSection8.1.1). Tasksthatexploitthreadconfinement.SingleͲthreadedexecutorsmakestrongerpromisesaboutconcurrencythando arbitrarythreadpools.Theyguaranteethattasksarenotexecutedconcurrently,whichallowsyoutorelaxthethread safetyoftaskcode.Objectscanbeconfinedtothetaskthread,thusenablingtasksdesignedtoruninthatthreadto access those objects without synchronization, even if those resources are not threadͲsafe. This forms an implicit couplingbetweenthetaskandtheexecutionpolicyͲthetasksrequiretheirexecutortobesingleͲthreaded.[1]Inthis case,ifyouchangedtheExecutorfromasingleͲthreadedonetoathreadpool,threadsafetycouldbelost. [1] The requirement is not quite this strong; it would be enough to ensure only that tasks not execute concurrently and provide enough synchronizationsothatthememoryeffectsofonetaskareguaranteedtobevisibletothenexttaskͲwhichispreciselytheguaranteeofferedby newSingle-ThreadExecutor. ResponseͲtimeͲsensitive tasks. GUI applications are sensitive to response time: users are annoyed at long delays between a button click and the corresponding visual feedback. Submitting a longͲrunning task to a singleͲthreaded executor,orsubmittingseverallongͲrunningtaskstoathreadpoolwithasmallnumberofthreads,mayimpairthe responsivenessoftheservicemanagedbythatExecutor. TasksthatuseThreadLocal.ThreadLocalallowseachthreadtohaveitsownprivate"version"ofavariable.However, executorsarefreetoreusethreadsastheyseefit.ThestandardExecutorimplementationsmayreapidlethreadswhen demand is low and add new ones when demand is high, and also replace a worker thread with a fresh one if an uncheckedexceptionisthrownfromatask.ThreadLocalmakessensetouseinpoolthreadsonlyifthethreadͲlocal valuehasalifetimethatisboundedbythatofatask;Thread-Localshouldnotbeusedinpoolthreadstocommunicate valuesbetweentasks. Threadpoolsworkbestwhentasksarehomogeneousandindependent.MixinglongͲrunningandshortͲrunningtasks risks"clogging"thepoolunlessitisverylarge;submittingtasksthatdependonothertasksrisksdeadlockunlessthe poolisunbounded.Fortunately,requestsintypicalnetworkͲbasedserverapplicationsͲwebservers,mailservers,file serversͲusuallymeettheseguidelines. Sometaskshavecharacteristicsthatrequireorprecludeaspecificexecutionpolicy.Tasksthatdependonothertasks require that the thread pool be large enough that tasks are never queued or rejected; tasks that exploit thread confinementrequiresequentialexecution.Documenttheserequirementssothatfuturemaintainersdonotundermine safetyorlivenessbysubstitutinganincompatibleexecutionpolicy. 8.1.1.ThreadStarvationDeadlock Iftasksthatdependonothertasksexecuteinathreadpool,theycandeadlock.InasingleͲthreadedexecutor,atask thatsubmitsanothertasktothesameexecutorandwaitsforitsresultwillalwaysdeadlock.Thesecondtasksitsonthe work queue until the first task completes, but the first will not complete because it is waiting for the result of the secondtask.Thesamethingcanhappeninlargerthreadpoolsifallthreadsareexecutingtasksthatareblockedwaiting forothertasksstillontheworkqueue.Thisiscalledthreadstarvationdeadlock,andcanoccurwheneverapooltask initiates an unbounded blocking wait for some resource or condition that can succeed only through the action of  1055BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools anotherpooltask,suchaswaitingforthereturnvalueorsideeffectofanothertask,unlessyoucanguaranteethatthe poolislargeenough. ThreadDeadlockinListing8.1illustratesthreadstarvationdeadlock.Render-PageTasksubmitstwoadditionaltasksto theExecutortofetchthepageheaderandfooter,rendersthepagebody,waitsfortheresultsoftheheaderandfooter tasks, and then combines the header, body, and footer into the finished page. With a singleͲthreaded executor, ThreadDeadlockwillalwaysdeadlock.Similarly,taskscoordinatingamongstthemselveswithabarriercouldalsocause threadstarvationdeadlockifthepoolisnotbigenough. WheneveryousubmittoanExecutortasksthatarenotindependent,beawareofthepossibilityofthreadstarvation deadlock, and document any pool sizing or configuration constraints in the code or configuration file where the Executorisconfigured. Inadditiontoanyexplicitboundsonthesizeofathreadpool,theremayalsobeimplicitlimitsbecauseofconstraintson otherresources.IfyourapplicationusesaJDBCconnectionpoolwithtenconnectionsandeachtaskneedsadatabase connection, it is as if your thread pool only has ten threads because tasks in excess of ten will block waiting for a connection. Listing8.1.TaskthatDeadlocksinaSingleǦthreadedExecutor.Don'tDothis. public class ThreadDeadlock { ExecutorService exec = Executors.newSingleThreadExecutor(); public class RenderPageTask implements Callable { public String call() throws Exception { Future header, footer; header = exec.submit(new LoadFileTask("header.html")); footer = exec.submit(new LoadFileTask("footer.html")); String page = renderBody(); // Will deadlock -- task waiting for result of subtask return header.get() + page + footer.get(); } } } 8.1.2.LongǦrunningTasks Threadpoolscanhaveresponsivenessproblemsiftaskscanblockforextendedperiodsoftime,evenifdeadlockisnota possibility.AthreadpoolcanbecomecloggedwithlongͲrunningtasks,increasingtheservicetimeevenforshorttasks.If thepoolsizeistoosmallrelativetotheexpectedsteadyͲstatenumberoflongrunningtasks,eventuallyallthepool threadswillberunninglongͲrunningtasksandresponsivenesswillsuffer. OnetechniquethatcanmitigatetheilleffectsoflongͲrunningtasksisfortaskstousetimedresourcewaitsinsteadof unboundedwaits.Mostblockingmethodsintheplaformlibrariescomeinbothuntimedandtimedversions,suchas Thread.join,BlockingQueue.put,CountDownLatch.await,andSelector.select.Ifthewaittimesout,youcanmark thetaskasfailedandabortitorrequeueitforexecutionlater.Thisguaranteesthateachtaskeventuallymakesprogress towards either successful or failed completion, freeing up threads for tasks that might complete more quickly. If a threadpoolisfrequentlyfullofblockedtasks,thismayalsobeasignthatthepool 8.2.SizingThreadPools Theidealsizeforathreadpooldependsonthetypesoftasksthatwillbesubmittedandthecharacteristicsofthe deployment system. Thread pool sizes should rarely be hardͲcoded; instead pool sizes should be provided by a configurationmechanismorcomputeddynamicallybyconsultingRuntime.availableProcessors. Sizingthreadpoolsisnotanexactscience,butfortunatelyyouneedonlyavoidtheextremesof"toobig"and"too small".Ifa threadpoolistoobig,thenthreads competeforscarceCPUandmemoryresources,resultinginhigher memoryusageandpossibleresourceexhaustion.Ifitistoosmall,throughputsuffersasprocessorsgounuseddespite availablework. Tosizeathreadpoolproperly,youneedtounderstandyourcomputingenvironment,yourresourcebudget,andthe natureofyourtasks.Howmanyprocessorsdoesthedeploymentsystemhave?Howmuchmemory?Dotasksperform .candidateforreapingandcanbeterminatedifthecurrentpoolsizeexceedsthecoresize howmanypoolthreadscanbeactiveatonce.AthreadthathasbeenidleforlongerthanthekeepͲalivetimebecomesa andwillnotcreatemorethreadsthanthisunlesstheworkqueueisfull.[3]Themaximumpoolsizeistheupperboundon targetsize;theimplementationattemptstomaintainthepoolatthissizeevenwhentherearenotaskstoexecute,[2] Thecorepoolsize,maximumpoolsize,andkeepͲalivetimegovernthreadcreationandteardown.Thecoresizeisthe 8.3.1.ThreadCreationandTeardown generalofwhichisshowninListing8.2. forthedefaultconfigurationsandusethemasastartingpoint.ThreadPoolExecutorhasseveralconstructors,themost constructorandcustomizeitasyouseefit;youcanconsultthesourcecodeforExecutorstoseetheexecutionpolicies If the default execution policy does not meet your needs, you can instantiate a ThreadPoolExecutor through its robustpoolimplementationthatallowsavarietyofcustomizations. newFixedThreadPool,andnewScheduled-ThreadExecutorfactoriesinExecutors.ThreadPoolExecutorisaflexible, ThreadPoolExecutor provides the base implementation for the executors returned by the newCachedThreadPool, 8.3.ConfiguringThreadPoolExecutor thethreadpoolsize. Similarly,whentheonlyconsumersofconnectionsarepooltasks,theeffectivesizeoftheconnectionpoolislimitedby other.Ifeachtaskrequiresaconnection,theeffectivesizeofthethreadpoolislimitedbytheconnectionpoolsize. Whentasksrequireapooledresourcesuchasdatabaseconnections,threadpoolsizeandresourcepoolsizeaffecteach dividethatintothetotalquantityavailable.Theresultwillbeanupperboundonthepoolsize. sizeconstraintsforthesetypesofresourcesiseasier:justadduphowmuchofthatresourceeachtaskrequiresand contributetosizingconstraintsarememory,filehandles,sockethandles,anddatabaseconnections.Calculatingpool Ofcourse,CPUcyclesarenottheonlyresourceyoumightwanttomanageusingthreadpools.Otherresourcesthatcan int N_CPUS = Runtime.getRuntime().availableProcessors(); YoucandeterminethenumberofCPUsusingRuntime: ൰ ܥ ൬ͳ൅ܹ כ ௖௣௨ܷכ ܰ௧௛௥௘௔ௗ௦ ൌܰ௖௣௨ Theoptimalpoolsizeforkeepingtheprocessorsatthedesiredutilizationis: ratio of wait time to compute time = ܥ ܹ ܷ௖௣௨ = target CPU utilization, Ͳ൑ܷ௖௣௨ ൑ͳ ܰ௖௣௨ = number of CPUs Giventhesedefinitions: CPUutilization. betunedbyrunningtheapplicationusingseveraldifferentpoolsizesunderabenchmarkloadandobservingthelevelof notbepreciseandcanbeobtainedthroughproͲfilingorinstrumentation.Alternatively,thesizeofthethreadpoolcan sizethepoolproperly,youmustestimatetheratioofwaitingtimetocomputetimeforyourtasks;thisestimateneed otherblockingoperations,youwantalargerpool,sincenotallofthethreadswillbeschedulableatalltimes.Inorderto "extra"runnablethreadpreventsCPUcyclesfromgoingunusedwhenthishappens.)FortasksthatalsoincludeI/Oor +1 threads. (Even computeͲintensive threads occasionally take a page fault or pause for some other reason, so an ForcomputeͲintensivetasks,anNcpuͲprocessorsystemusuallyachievesoptimumutilizationwithathreadpoolofNcpu tunedaccordingtoitsworkload. havedifferentcategoriesoftaskswithverydifferentbehaviors,considerusingmultiplethreadpoolssoeachcanbe mostlycomputation,I/O,orsomecombination?Dotheyrequireascarceresource,suchasaJDBCconnection?Ifyou JavaConcurrencyInPractice 106   1075BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools [2]WhenaThreadPoolExecutorisinitiallycreated,thecorethreadsarenotstartedimmediatelybutinsteadastasksaresubmitted,unless youcallprestartAllCoreThreads. [3]Developersaresometimestemptedtosetthecoresizetozerosothattheworkerthreadswilleventuallybetorndownandthereforewon't preventtheJVMfromexiting,butthiscancausesomestrangeͲseemingbehaviorinthreadpoolsthatdon'tuseaSynchronousQueuefortheir workqueue(asnewCachedThreadPooldoes).Ifthepoolisalreadyatthecoresize,ThreadPoolExecutorcreatesanewthreadonlyif theworkqueueisfull.Sotaskssubmittedtoathreadpoolwithaworkqueuethathasanycapacityandacoresizeofzerowillnotexecuteuntil thequeuefillsup,whichisusuallynotwhatisdesired.InJava6,allowCoreThreadTimeOutallowsyoutorequestthatallpoolthreadsbe abletotimeout;enablethisfeaturewithacoresizeofzeroifyouwantaboundedthreadpoolwithaboundedworkqueuebutstillhaveallthe threadstorndownwhenthereisnoworktodo. Listing8.2.GeneralConstructorforThreadPoolExecutor. public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) { ... } BytuningthecorepoolsizeandkeepͲalivetimes,youcanencouragethepooltoreclaimresourcesusedbyotherwise idlethreads,makingthemavailableformoreusefulwork.(Likeeverythingelse,thisisatradeoff:reapingidlethreads incursadditionallatencyduetothreadcreationifthreadsmustlaterbecreatedwhendemandincreases.) ThenewFixedThreadPoolfactorysetsboththecorepoolsizeandthemaximumpoolsizetotherequestedpoolsize, creating the effect of infinite timeout; the newCachedThreadPool factory sets the maximum pool size to Integer.MAX_VALUEandthecorepoolsizetozerowithatimeoutofoneminute,creatingtheeffectofaninfinitely expandablethreadpoolthatwillcontractagainwhendemanddecreases.Othercombinationsarepossibleusingthe explicitThreadPool-Executorconstructor. 8.3.2.ManagingQueuedTasks Boundedthreadpoolslimitthenumberoftasksthatcanbeexecutedconcurrently.(ThesingleͲthreadedexecutorsarea notablespecialcase:theyguaranteethatnotaskswillexecuteconcurrently,offeringthepossibilityofachievingthread safetythroughthreadconfinement.) WesawinSection6.1.2howunboundedthreadcreationcouldleadtoinstability,andaddressedthisproblembyusinga fixedͲsizedthreadpoolinsteadofcreatinganewthreadforeveryrequest.However,thisisonlyapartialsolution;itis stillpossiblefortheapplicationtorunoutofresourcesunderheavyload,justharder.Ifthearrivalratefornewrequests exceedstherateatwhichtheycanbehandled,requestswillstillqueueup.Withathreadpool,theywaitinaqueueof RunnablesmanagedbytheExecutorinsteadofqueueingupasthreadscontendingfortheCPU.Representingawaiting taskwithaRunnableandalistnodeiscertainlyalotcheaperthanwithathread,buttheriskofresourceexhaustionstill remainsifclientscanthrowrequestsattheserverfasterthanitcanhandlethem. Requests often arrive in bursts even when the average request rate is fairly stable. Queues can help smooth out transientburstsoftasks,butiftaskscontinuetoarrivetooquicklyyouwilleventuallyhavetothrottlethearrivalrateto avoidrunningoutofmemory.[4]Evenbeforeyourunoutofmemory,responsetimewillgetprogressivelyworseasthe taskqueuegrows. [4]Thisisanalogoustoflowcontrolincommunicationsnetworks:youmaybewillingtobufferacertainamountofdata,buteventuallyyouneed tofindawaytogettheothersidetostopsendingyoudata,orthrowtheexcessdataonthefloorandhopethesenderretransmitsitwhenyou're notsobusy. ThreadPoolExecutorallowsyoutosupplya BlockingQueuetoholdtasksawaitingexecution.Therearethreebasic approaches to task queuing: unbounded queue, bounded queue, and synchronous handoff. The choice of queue interactswithotherconfigurationparameterssuchaspoolsize. ThedefaultfornewFixedThreadPoolandnewSingleThreadExecutoristouseanunboundedLinkedBlockingQueue. Taskswillqueueupifallworkerthreadsarebusy,butthequeuecouldgrowwithoutboundifthetaskskeeparriving fasterthantheycanbeexecuted. A more stable resource management strategy is to use a bounded queue, such as an ArrayBlockingQueue or a boundedLinkedBlockingQueueorPriority-BlockingQueue.Boundedqueueshelppreventresourceexhaustionbut introducethequestionofwhattodowithnewtaskswhenthequeueisfull.(Thereareanumberofpossiblesaturation policiesforaddressingthisproblem;seeSection8.3.3.)Withaboundedworkqueue,thequeuesizeandpoolsizemust betunedtogether.Alargequeuecoupledwithasmallpoolcanhelpreducememoryusage,CPUusage,andcontext switching,atthecostofpotentiallyconstrainingthroughput.  108 JavaConcurrencyInPractice For very large or unbounded pools, you can also bypass queuing entirely and instead hand off tasks directly from producers to worker threads using a SynchronousQueue. A SynchronousQueue is not really a queue at all, but a mechanism for managing handoffs between threads. In order to put an element on a SynchronousQueue, another threadmustalreadybewaitingtoacceptthehandoff.Ifnothreadiswaitingbutthecurrentpoolsizeislessthanthe maximum, Thread-PoolExecutor creates a new thread; otherwise the task is rejected according to the saturation policy.Usingadirecthandoffismoreefficientbecausethetaskcanbehandedrighttothethreadthatwillexecuteit, ratherthanfirstplacingitonaqueueandthenhavingtheworkerthreadfetchitfromthequeue.SynchronousQueueis apracticalchoiceonlyifthepoolisunboundedorifrejectingexcesstasksisacceptable.The newCachedThreadPool factoryusesaSynchronousQueue. UsingaFIFOqueuelikeLinkedBlockingQueueorArrayBlockingQueuecausestaskstobestartedintheorderinwhich theyarrived.Formorecontrolovertaskexecutionorder,youcanuseaPriorityBlockingQueue,whichorderstasks accordingtopriority.Prioritycanbedefinedbynaturalorder(iftasksimplementComparable)orbyaComparator. The newCachedThreadPoolfactoryisagooddefaultchoicefor an Executor,providingbetterqueuingperformance thanafixedthreadpool.[5]Afixedsizethreadpoolisagoodchoicewhenyouneedtolimitthenumberofconcurrent tasksforresourceͲmanagementpurposes,asinaserverapplicationthatacceptsrequestsfromnetworkclientsand wouldotherwisebevulnerabletooverload. [5]ThisperformancedifferencecomesfromtheuseofSynchronousQueueinsteadofLinkedBlocking-Queue.SynchronousQueue wasreplacedinJava6withanewnonͲblockingalgorithmthatimprovedthroughputinExecutorbenchmarksbyafactorofthreeovertheJava 5.0SynchronousQueueimplementation(Schereretal.,2006). Boundingeitherthethreadpoolortheworkqueueissuitableonlywhentasksareindependent.Withtasksthatdepend onothertasks,boundedthreadpoolsorqueuescancausethreadstarvationdeadlock;instead,useanunboundedpool configurationlikenewCachedThreadPool.[6] [6] An alternative configuration for tasks that submit other tasks and wait for their results is to use a bounded thread pool, a SynchronousQueueastheworkqueue,andthecallerͲrunssaturationpolicy. 8.3.3.SaturationPolicies When a bounded work queue fills up, the saturation policy comes into play. The saturation policy for a ThreadPoolExecutorcanbemodifiedbycalling setRejectedExecutionHandler.(Thesaturationpolicyisalsoused when a task is submitted to an Executor that has been shut down.) Several implementations of RejectedExecutionHandler are provided, each implementing a different saturation policy: AbortPolicy, CallerRunsPolicy,DiscardPolicy,andDiscardOldestPolicy. Thedefaultpolicy,abort,causesexecutetothrowtheuncheckedRejected-ExecutionException;thecallercancatch thisexceptionandimplementitsownoverflowhandlingasitseesfit.Thediscardpolicysilentlydiscardsthenewly submittedtaskifitcannotbequeuedforexecution;thediscardͲoldestpolicydiscardsthetaskthatwouldotherwisebe executednextandtriestoresubmitthenewtask.(Iftheworkqueueisapriorityqueue,thisdiscardsthehighestͲ priorityelement,sothecombinationofadiscardͲoldestsaturationpolicyandapriorityqueueisnotagoodone.) ThecallerͲrunspolicyimplementsaformofthrottlingthatneitherdiscardstasksnorthrowsanexception,butinstead tries to slow down the flow of new tasks by pushing some of the work back to the caller. It executes the newly submittedtasknotinapoolthread,butinthethreadthatcallsexecute.IfwemodifiedourWebServerexampletouse aboundedqueueandthecallerͲrunspolicy,afterallthepoolthreadswereoccupiedandtheworkqueuefilledupthe nexttaskwouldbeexecutedinthemainthreadduringthecalltoexecute.Sincethiswouldprobablytakesometime, themainthreadcannotsubmitanymoretasksforatleastalittlewhile,givingtheworkerthreadssometimetocatch uponthebacklog.Themainthreadwouldalsonotbecallingacceptduringthistime,soincomingrequestswillqueue upintheTCPlayerinsteadofintheapplication.Iftheoverloadpersisted,eventuallytheTCPlayerwoulddecideithas queued enough connection requests and begin discarding connection requests as well. As the server becomes overloaded,theoverloadisgraduallypushedoutwardͲfromthepoolthreadstotheworkqueuetotheapplicationto theTCPlayer,andeventuallytotheclientͲenablingmoregracefuldegradationunderload. Choosing a saturation policy or making other changes to the execution policy can be done when the Executor is created.Listing8.3illustratescreatingafixedsizethreadpoolwiththecallerͲrunssaturationpolicy.  1095BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools Listing8.3.CreatingaFixedǦsizedThreadPoolwithaBoundedQueueandtheCallerǦrunsSaturationPolicy. ThreadPoolExecutor executor = new ThreadPoolExecutor(N_THREADS, N_THREADS, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue(CAPACITY)); executor.setRejectedExecutionHandler( new ThreadPoolExecutor.CallerRunsPolicy()); Thereisnopredefinedsaturationpolicytomakeexecuteblockwhentheworkqueueisfull.However,thesameeffect canbeaccomplishedbyusingaSemaphoretoboundthetaskinjectionrate,asshowninBoundedExecutorinListing8.4. Insuchanapproach,useanunboundedqueue(there'snoreasontoboundboththequeuesizeandtheinjectionrate) andsettheboundonthesemaphoretobeequaltothepoolsizeplusthenumberofqueuedtasksyouwanttoallow, sincethesemaphoreisboundingthenumberoftasksbothcurrentlyexecutingandawaitingexecution. 8.3.4.ThreadFactories Wheneverathreadpoolneedstocreateathread,itdoessothroughathreadfactory(seeListing8.5).Thedefault threadfactorycreatesanew,nondaemonthreadwithnospecialconfiguration.Specifyingathreadfactoryallowsyouto customizetheconfigurationofpoolthreads.ThreadFactoryhasasinglemethod,newThread,thatiscalledwhenevera threadpoolneedstocreateanewthread. There are a number of reasons to use a custom thread factory. You might want to specify an UncaughtExceptionHandlerforpoolthreads,orinstantiateaninstanceofacustom Threadclass,suchasonethat performsdebuglogging.Youmightwanttomodifythepriority(generallynotaverygoodidea;seeSection10.3.1)or setthedaemonstatus(again,notallthatgoodanidea;seeSection7.4.2)ofpoolthreads.Ormaybeyoujustwantto givepoolthreadsmoremeaningfulnamestosimplifyinterpretingthreaddumpsanderrorlogs. Listing8.4.UsingaSemaphoretoThrottleTaskSubmission. @ThreadSafe public class BoundedExecutor { private final Executor exec; private final Semaphore semaphore; public BoundedExecutor(Executor exec, int bound) { this.exec = exec; this.semaphore = new Semaphore(bound); } public void submitTask(final Runnable command) throws InterruptedException { semaphore.acquire(); try { exec.execute(new Runnable() { public void run() { try { command.run(); } finally { semaphore.release(); } } }); } catch (RejectedExecutionException e) { semaphore.release(); } } } Listing8.5.ThreadFactoryInterface. public interface ThreadFactory { Thread newThread(Runnable r); } MyThreadFactoryinListing8.6illustratesacustomthreadfactory.ItinstantiatesanewMyAppThread,passingapoolͲ specificnametotheconstructorsothatthreadsfromeachpoolcanbedistinguishedinthreaddumpsanderrorlogs. My-AppThreadcanalsobeusedelsewhereintheapplicationsothatallthreadscantakeadvantageofitsdebugging features.  110 JavaConcurrencyInPractice Listing8.6.CustomThreadFactory. public class MyThreadFactory implements ThreadFactory { private final String poolName; public MyThreadFactory(String poolName) { this.poolName = poolName; } public Thread newThread(Runnable runnable) { return new MyAppThread(runnable, poolName); } } TheinterestingcustomizationtakesplaceinMyAppThread,showninListing8.7,whichletsyouprovideathreadname, sets a custom UncaughtException-Handler that writes a message to a Logger, maintains statistics on how many threadshavebeencreatedanddestroyed,andoptionallywritesadebugmessagetothelogwhenathreadiscreatedor terminates. Ifyourapplicationtakesadvantageofsecuritypoliciestograntpermissionstoparticularcodebases,youmaywantto use the privilegedThreadFactory factory method in Executors to construct your thread factory. It creates pool threadsthathavethesamepermissions,AccessControlContext,andcontextClassLoaderasthethreadcreatingthe privilegedThreadFactory.Otherwise,threadscreatedbythethreadpoolinheritpermissionsfromwhateverclient happenstobecallingexecuteorsubmitatthetimeanewthreadisneeded,whichcouldcauseconfusingsecurityͲ relatedexceptions. 8.3.5.CustomizingThreadPoolExecutorAfterConstruction MostoftheoptionspassedtotheThreadPoolExecutorconstructorscanalsobemodifiedafterconstructionviasetters (suchasthecorethreadpoolsize,maximumthreadpoolsize,keepͲalivetime,threadfactory,andrejectedexecution handler). If the Executor is created through one of the factory methods in Executors (except newSingleThreadExecutor),youcancasttheresulttoThread-PoolExecutortoaccessthesettersasinListing8.8. Executors includes a factory method, unconfigurableExecutorService, which takes an existing ExecutorService andwrapsitwithoneexposingonlythemethodsofExecutorServicesoitcannotbefurtherconfigured.Unlikethe pooledimplementations,newSingleThreadExecutorreturnsanExecutorServicewrappedinthismanner,ratherthan a raw ThreadPoolExecutor. While a singleͲthreaded executor is actually implemented as a thread pool with one thread,italsopromisesnottoexecutetasksconcurrently.Ifsomemisguidedcodeweretoincreasethepoolsizeona singleͲthreadedexecutor,itwouldunderminetheintendedexecutionsemantics.  1115BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools Listing8.7.CustomThreadBaseClass. public class MyAppThread extends Thread { public static final String DEFAULT_NAME = "MyAppThread"; private static volatile boolean debugLifecycle = false; private static final AtomicInteger created = new AtomicInteger(); private static final AtomicInteger alive = new AtomicInteger(); private static final Logger log = Logger.getAnonymousLogger(); public MyAppThread(Runnable r) { this(r, DEFAULT_NAME); } public MyAppThread(Runnable runnable, String name) { super(runnable, name + "-" + created.incrementAndGet()); setUncaughtExceptionHandler( new Thread.UncaughtExceptionHandler() { public void uncaughtException(Thread t, Throwable e) { log.log(Level.SEVERE, "UNCAUGHT in thread " + t.getName(), e); } }); } public void run() { // Copy debug flag to ensure consistent value throughout. boolean debug = debugLifecycle; if (debug) log.log(Level.FINE, "Created "+getName()); try { alive.incrementAndGet(); super.run(); } finally { alive.decrementAndGet(); if (debug) log.log(Level.FINE, "Exiting "+getName()); } } public static int getThreadsCreated() { return created.get(); } public static int getThreadsAlive() { return alive.get(); } public static boolean getDebug() { return debugLifecycle; } public static void setDebug(boolean b) { debugLifecycle = b; } } Listing8.8.ModifyinganExecutorCreatedwiththeStandardFactories. ExecutorService exec = Executors.newCachedThreadPool(); if (exec instanceof ThreadPoolExecutor) ((ThreadPoolExecutor) exec).setCorePoolSize(10); else throw new AssertionError("Oops, bad assumption"); Youcanusethistechniquewithyourownexecutorstopreventtheexecutionpolicyfrombeingmodified.Ifyouwillbe exposing an ExecutorService to code you don't trust not to modify it, you can wrap it with an unconfigurableExecutorService. 8.4.ExtendingThreadPoolExecutor ThreadPoolExecutorwasdesignedforextension,providingseveral"hooks"forsubclassestooverridebeforeExecute, afterExecute,andterminatethatcanbeusedtoextendthebehaviorofThreadPoolExecutor. The beforeExecute and afterExecute hooks are called in the thread that executes the task, and can be used for addinglogging,timing,monitoring,orstatisticsgathering.TheafterExecutehookiscalledwhetherthetaskcompletes byreturningnormallyfromrunorbythrowinganException.(IfthetaskcompleteswithanError,afterExecuteisnot called.)IfbeforeExecutethrowsaRuntimeException,thetaskisnotexecutedandafterExecuteisnotcalled. Theterminatedhookiscalledwhenthethreadpoolcompletestheshutdownprocess,afteralltaskshavefinishedand allworkerthreadshaveshutdown.ItcanbeusedtoreleaseresourcesallocatedbytheExecutorduringitslifecycle, performnotificationorlogging,orfinalizestatisticsgathering. 8.4.1.Example:AddingStatisticstoaThreadPool TimingThreadPool in Listing 8.9 shows a custom thread pool that uses before-Execute, afterExecute, and terminatedtoaddloggingandstatisticsgathering.Tomeasureatask'sruntime,beforeExecutemustrecordthestart timeandstoreitsomewhereafterExecutecanfindit.Becauseexecutionhooksarecalledinthethreadthatexecutes thetask,avalueplacedinaThreadLocalbybeforeExecutecanberetrievedbyafterExecute.TimingThreadPool usesapairofAtomicLongstokeeptrackofthetotalnumberoftasksprocessedandthetotalprocessingtime,anduses theterminatedhooktoprintalogmessageshowingtheaveragetasktime.  112 JavaConcurrencyInPractice Listing8.9.ThreadPoolExtendedwithLoggingandTiming. public class TimingThreadPool extends ThreadPoolExecutor { private final ThreadLocal startTime = new ThreadLocal(); private final Logger log = Logger.getLogger("TimingThreadPool"); private final AtomicLong numTasks = new AtomicLong(); private final AtomicLong totalTime = new AtomicLong(); protected void beforeExecute(Thread t, Runnable r) { super.beforeExecute(t, r); log.fine(String.format("Thread %s: start %s", t, r)); startTime.set(System.nanoTime()); } protected void afterExecute(Runnable r, Throwable t) { try { long endTime = System.nanoTime(); long taskTime = endTime - startTime.get(); numTasks.incrementAndGet(); totalTime.addAndGet(taskTime); log.fine(String.format("Thread %s: end %s, time=%dns", t, r, taskTime)); } finally { super.afterExecute(r, t); } } protected void terminated() { try { log.info(String.format("Terminated: avg time=%dns", totalTime.get() / numTasks.get())); } finally { super.terminated(); } } } 8.5.ParallelizingRecursiveAlgorithms ThepagerenderingexamplesinSection6.3wentthroughaseriesofrefinementsinsearchofexploitableparallelism. Thefirstattemptwasentirelysequential;thesecondusedtwothreadsbutstillperformedalltheimagedownloads sequentially;thefinalversiontreatedeachimagedownloadasaseparatetasktoachievegreaterparallelism.Loops whosebodiescontainnontrivialcomputationorperformpotentiallyblockingI/Oarefrequentlygoodcandidatesfor parallelization,aslongastheiterationsareindependent. Ifwehavealoopwhoseiterationsareindependentandwe don'tneed towaitforall ofthemtocompletebefore proceeding, we can use an Executor to transform a sequential loop into a parallel one, as shown in processSequentiallyandprocessInParallelinListing8.10. Listing8.10.TransformingSequentialExecutionintoParallelExecution. void processSequentially(List elements) { for (Element e : elements) process(e); } void processInParallel(Executor exec, List elements) { for (final Element e : elements) exec.execute(new Runnable() { public void run() { process(e); } }); } AcalltoprocessInParallelreturnsmorequicklythanacalltoprocessSequentiallybecauseitreturnsassoonasall thetasksarequeuedtotheExecutor,ratherthanwaitingforthemalltocomplete.Ifyouwanttosubmitasetoftasks andwaitforthemalltocomplete,youcanuseExecutorService.invokeAll;toretrievetheresultsastheybecome available,youcanuseaCompletionService,asinRendereronpage130. Sequentialloopiterationsaresuitableforparallelizationwheneachiterationisindependentoftheothersandthework doneineachiterationoftheloopbodyissignificantenoughtooffsetthecostofmanaginganewtask. Loopparallelizationcanalsobeappliedtosomerecursivedesigns;thereareoftensequentialloopswithintherecursive algorithmthatcanbeparallelizedinthesamemannerasListing8.10.Theeasiercaseiswheneachiterationdoesnot require the results of the recursive iterations it invokes. For example, sequentialRecursive in Listing 8.11 does a  1135BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools depthͲfirst traversal of a tree, performing a calculation on each node and placing the result in a collection. The transformedversion,parallelRecursive,alsodoesadepthͲfirsttraversal,butinsteadofcomputingtheresultaseach nodeisvisited,itsubmitsatasktocomputethenoderesult. Listing8.11.TransformingSequentialTailǦrecursionintoParallelizedRecursion. public void sequentialRecursive(List> nodes, Collection results) { for (Node n : nodes) { results.add(n.compute()); sequentialRecursive(n.getChildren(), results); } } public void parallelRecursive(final Executor exec, List> nodes, final Collection results) { for (final Node n : nodes) { exec.execute(new Runnable() { public void run() { results.add(n.compute()); } }); parallelRecursive(exec, n.getChildren(), results); } } WhenparallelRecursivereturns,eachnodeinthetreehasbeenvisited(thetraversalisstillsequential:onlythecalls tocomputeareexecutedinparallel)andthecomputationforeachnodehasbeenqueuedtotheExecutor.Callersof parallelRecursivecanwaitforalltheresultsbycreatinganExecutorspecifictothetraversalandusingshutdown andawaitTermination,asshowninListing8.12. Listing8.12.WaitingforResultstobeCalculatedinParallel. public Collection getParallelResults(List> nodes) throws InterruptedException { ExecutorService exec = Executors.newCachedThreadPool(); Queue resultQueue = new ConcurrentLinkedQueue(); parallelRecursive(exec, nodes, resultQueue); exec.shutdown(); exec.awaitTermination(Long.MAX_VALUE, TimeUnit.SECONDS); return resultQueue; } 8.5.1.Example:APuzzleFramework Anappealingapplicationofthistechniqueissolvingpuzzlesthatinvolvefindingasequenceoftransformationsfrom someinitialstatetoreachagoalstate,suchasthefamiliar"slidingblockpuzzles",[7]"HiͲQ","InstantInsanity",andother solitairepuzzles. [7]Seehttp://www.puzzleworld.org/SlidingBlockPuzzles. Wedefinea"puzzle"asacombinationofaninitialposition,agoalposition,andasetofrulesthatdeterminevalid moves.Therulesethastwoparts:computingthelistoflegalmovesfromagivenpositionandcomputingtheresultof applying a move to a position. Puzzle in Listing 8.13 shows our puzzle abstraction; the type parameters P and M represent the classes for a position and a move. From this interface, we can write a simple sequential solver that searchesthepuzzlespaceuntilasolutionisfoundorthepuzzlespaceisexhausted. Listing8.13.AbstractionforPuzzlesLikethe"SlidingBlocksPuzzle". public interface Puzzle { P initialPosition(); boolean isGoal(P position); Set legalMoves(P position); P move(P position, M move); } NodeinListing8.14representsapositionthathasbeenreachedthroughsomeseriesofmoves,holdingareferenceto themovethatcreatedthepositionandthepreviousNode.FollowingthelinksbackfromaNodeletsusreconstructthe sequenceofmovesthatledtothecurrentposition. SequentialPuzzleSolverinListing8.15showsasequentialsolverforthepuzzleframeworkthatperformsadepthͲfirst searchofthepuzzlespace.Itterminateswhenitfindsasolution(whichisnotnecessarilytheshortestsolution). Rewritingthesolvertoexploitconcurrencywouldallowustocomputenextmovesandevaluatethegoalconditionin parallel,sincetheprocessofevaluatingonemoveismostlyindependentofevaluatingothermoves.(Wesay"mostly"  114 JavaConcurrencyInPractice becausetaskssharesomemutablestate,suchasthesetofseenpositions.)Ifmultipleprocessorsareavailable,this couldreducethetimeittakestofindasolution. ConcurrentPuzzleSolverinListing8.16usesaninnerSolverTaskclassthatextendsNodeandimplementsRunnable. Most of the work is done in run: evaluating the set of possible next positions, pruning positions already searched, evaluatingwhethersuccesshasyetbeenachieved(by this taskorbysome othertask), andsubmittingunsearched positionstoanExecutor. To avoid infinite loops, the sequential version maintained a Set of previously searched positions; ConcurrentPuzzleSolverusesaConcurrentHashMapforthispurpose.Thisprovidesthreadsafetyandavoidstherace conditioninherentinconditionallyupdatingasharedcollectionbyusingputIfAbsenttoatomicallyaddapositiononly ifitwasnotpreviouslyknown.ConcurrentPuzzleSolverusestheinternalworkqueueofthethreadpoolinsteadof thecallstacktoholdthestateofthesearch. Listing8.14.LinkNodeforthePuzzleSolverFramework. @Immutable static class Node { final P pos; final M move; final Node prev; Node(P pos, M move, Node prev) {...} List asMoveList() { List solution = new LinkedList(); for (Node n = this; n.move != null; n = n.prev) solution.add(0, n.move); return solution; } } Theconcurrentapproachalsotradesoneformoflimitationforanotherthatmightbemoresuitabletotheproblem domain.ThesequentialversionperformsadepthͲfirstsearch,sothesearchisboundedbytheavailablestacksize.The concurrentversionperformsabreadthͲfirstsearchandisthereforefreeofthestacksizerestriction(butcanstillrunout ofmemoryifthesetofpositionstobesearchedoralreadysearchedexceedstheavailablememory). In order to stop searching when we find a solution, we need a way to determine whether any thread has found a solutionyet.Ifwewanttoacceptthefirstsolutionfound,wealsoneedtoupdatethesolutiononlyifnoothertaskhas alreadyfoundone.Theserequirementsdescribeasortoflatch(seeSection5.5.1)andinparticular,aresultͲbearing latch.WecouldeasilybuildablockingresultbearinglatchusingthetechniquesinChapter14,butitisofteneasierand lesserrorͲpronetouseexistinglibraryclassesratherthanlowͲlevellanguagemechanisms.ValueLatchinListing8.17 usesaCountDownLatchtoprovidetheneededlatchingbehavior,anduseslockingtoensurethatthesolutionissetonly once. Eachtaskfirstconsultsthesolutionlatchandstopsifasolutionhasalreadybeenfound.Themainthreadneedstowait untilasolutionisfound;getValueinValueLatchblocksuntilsomethreadhassetthevalue.ValueLatchprovidesa waytoholdavaluesuchthatonlythefirstcallactuallysetsthevalue,callerscantestwhetherithasbeenset,and callerscanblockwaitingforittobeset.OnthefirstcalltosetValue,thesolutionisupdatedandtheCountDownLatchis decremented,releasingthemainsolverthreadfromgetValue. ThefirstthreadtofindasolutionalsoshutsdowntheExecutor,topreventnewtasksfrombeingaccepted.Toavoid havingtodealwithRejectedExecutionException,therejectedexecutionhandlershouldbesettodiscardsubmitted tasks.Then,allunfinishedtaskseventuallyruntocompletionandanysubsequentattemptstoexecutenewtasksfail silently,allowingtheexecutortoterminate.(Ifthetaskstooklongertorun,wemightwanttointerrupttheminsteadof lettingthemfinish.)  1155BPartII:StructuringConcurrentApplications Ͳ 20BChapter8.ApplyingThreadPools Listing8.15.SequentialPuzzleSolver. public class SequentialPuzzleSolver { private final Puzzle puzzle; private final Set

seen = new HashSet

(); public SequentialPuzzleSolver(Puzzle puzzle) { this.puzzle = puzzle; } public List solve() { P pos = puzzle.initialPosition(); return search(new Node(pos, null, null)); } private List search(Node node) { if (!seen.contains(node.pos)) { seen.add(node.pos); if (puzzle.isGoal(node.pos)) return node.asMoveList(); for (M move : puzzle.legalMoves(node.pos)) { P pos = puzzle.move(node.pos, move); Node child = new Node(pos, move, node); List result = search(child); if (result != null) return result; } } return null; } static class Node { /* Listing 8.14 */ } } Listing8.16.ConcurrentVersionofPuzzleSolver. public class ConcurrentPuzzleSolver { private final Puzzle puzzle; private final ExecutorService exec; private final ConcurrentMap seen; final ValueLatch> solution = new ValueLatch>(); ... public List solve() throws InterruptedException { try { P p = puzzle.initialPosition(); exec.execute(newTask(p, null, null)); // block until solution found Node solnNode = solution.getValue(); return (solnNode == null) ? null : solnNode.asMoveList(); } finally { exec.shutdown(); } } protected Runnable newTask(P p, M m, Node n) { return new SolverTask(p, m, n); } class SolverTask extends Node implements Runnable { ... public void run() { if (solution.isSet() || seen.putIfAbsent(pos, true) != null) return; // already solved or seen this position if (puzzle.isGoal(pos)) solution.setValue(this); else for (M m : puzzle.legalMoves(pos)) exec.execute( newTask(puzzle.move(pos, m), m, this)); } } }    116 JavaConcurrencyInPractice Listing8.17.ResultǦbearingLatchUsedbyConcurrentPuzzleSolver. @ThreadSafe public class ValueLatch { @GuardedBy("this") private T value = null; private final CountDownLatch done = new CountDownLatch(1); public boolean isSet() { return (done.getCount() == 0); } public synchronized void setValue(T newValue) { if (!isSet()) { value = newValue; done.countDown(); } } public T getValue() throws InterruptedException { done.await(); synchronized (this) { return value; } } } ConcurrentPuzzleSolver does not deal well with the case where there is no solution: if all possible moves and positionshavebeenevaluatedandnosolutionhasbeenfound, solvewaitsforeverinthecalltogetSolution.The sequentialversionterminatedwhenithadexhaustedthesearchspace,butgettingconcurrentprogramstoterminate cansometimesbemoredifficult.Onepossiblesolutionistokeepacountofactivesolvertasksandsetthesolutionto nullwhenthecountdropstozero,asinListing8.18. Findingthesolutionmayalsotakelongerthanwearewillingtowait;thereareseveraladditionalterminationconditions wecouldimposeonthesolver.Oneisatimelimit;thisiseasilydonebyimplementingatimedgetValueinValueLatch (whichwouldusethetimedversionofawait),andshuttingdowntheExecutoranddeclaringfailureifgetValuetimes out.AnotherissomesortofpuzzleͲspecificmetricsuchassearchingonlyuptoacertainnumberofpositions.Orwecan provideacancellationmechanismandlettheclientmakeitsowndecisionaboutwhentostopsearching. Listing8.18.SolverthatRecognizeswhenNoSolutionExists. public class PuzzleSolver extends ConcurrentPuzzleSolver { ... private final AtomicInteger taskCount = new AtomicInteger(0); protected Runnable newTask(P p, M m, Node n) { return new CountingSolverTask(p, m, n); } class CountingSolverTask extends SolverTask { CountingSolverTask(P pos, M move, Node prev) { super(pos, move, prev); taskCount.incrementAndGet(); } public void run() { try { super.run(); } finally { if (taskCount.decrementAndGet() == 0) solution.setValue(null); } } } } Summary TheExecutorframeworkisapowerfulandflexibleframeworkforconcurrentlyexecutingtasks.Itoffersanumberof tuningoptions,suchaspoliciesforcreatingandtearingdownthreads,handlingqueuedtasks,andwhattodowith excesstasks,andprovidesseveralhooksforextendingitsbehavior.Asinmostpowerfulframeworks,however,there arecombinationsofsettingsthatdonotworkwelltogether;sometypesoftasksrequirespecificexecutionpolicies,and somecombinationsoftuningparametersmayproducestrangeresults.  1175BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications Chapter9.GUIApplications If you've tried to write even a simple GUI application using Swing, you know that GUI applications have their own peculiarthreadingissues.Tomaintainsafety,certaintasksmustrunintheSwingeventthread.Butyoucannotexecute longͲrunningtasksintheeventthread,lesttheUIbecomeunresponsive.AndSwingdatastructuresarenotthreadͲsafe, soyoumustbecarefultoconfinethemtotheeventthread. Nearly all GUI toolkits, including Swing and SWT, are implemented as singleͲthreaded subsystems in which all GUI activityisconfinedtoasinglethread.IfyouarenotplanningtowriteatotallysingleͲthreadedprogram,therewillbe activitiesthatrunpartiallyinanapplicationthreadandpartiallyintheeventthread.Likemanyotherthreadingbugs, gettingthisdivisionwrongmaynotnecessarilymakeyourprogramcrashimmediately;instead,itcouldbehaveoddly underhardͲtoͲidentifyconditions.EventhoughtheGUIframeworksthemselvesaresingleͲthreadedsubsystems,your applicationmaynotbe,andyoustillneedtoconsiderthreadingissuescarefullywhenwritingGUIcode. 9.1.WhyareGUIsSingleǦthreaded? In the old days, GUI applications were singleͲthreaded and GUI events were processed from a "main event loop". ModernGUIframeworksuseamodelthatisonlyslightlydifferent:theycreateadedicatedeventdispatchthread(EDT) forhandlingGUIevents. SingleͲthreadedGUIframeworksarenotuniquetoJava;Qt,NextStep,MacOSCocoa,XWindows,andmanyothersare also singleͲthreaded. This is not for lack of trying; there have been many attempts to write multithreaded GUI frameworks,butbecauseofpersistentproblemswithraceconditionsanddeadlock,theyalleventuallyarrivedatthe singleͲthreadedeventqueuemodelinwhichadedicatedthreadfetcheseventsoffaqueueanddispatchesthemto applicationͲdefinedeventhandlers.(AWToriginallytriedtosupportagreaterdegreeofmultithreadedaccess,andthe decisiontomakeSwingsingleͲthreadedwasbasedlargelyonexperiencewithAWT.) MultithreadedGUIframeworkstendtobeparticularlysusceptibletodeadlock,partiallybecauseoftheunfortunate interaction between input event processing and any sensible objectͲoriented modeling of GUI components. Actions initiatedbytheusertendto"bubbleup"fromtheOStotheapplicationͲamouseclickisdetectedbytheOS,isturned intoa"mouseclick"eventbythetoolkit,andiseventuallydeliveredtoanapplicationlistenerasahigherlevelevent suchasa"buttonpressed"event.Ontheotherhand,applicationͲinitiatedactions"bubbledown"fromtheapplication totheOSͲchangingthebackgroundcolorofacomponentoriginatesintheapplicationandisdispatchedtoaspecific componentclassandeventuallyintotheOSforrendering.CombiningthistendencyforactivitiestoaccessthesameGUI objectsintheoppositeorderwiththerequirementofmakingeachobjectthreadͲsafeyieldsarecipeforinconsistent lockordering,whichleadstodeadlock(seeChapter10).AndthisisexactlywhatnearlyeveryGUItoolkitdevelopment effortrediscoveredthroughexperience. AnotherfactorleadingtodeadlockinmultithreadedGUIframeworksistheprevalenceofthemodelͲviewͲcontrol(MVC) pattern.Factoringuserinteractionsintocooperatingmodel,view,andcontrollerobjectsgreatlysimplifiesimplementing GUI applications, but again raises the risk of inconsistent lock ordering. The controller calls into the model, which notifiestheviewthatsomethinghaschanged.Butthecontrollercanalsocallintotheview,whichmayinturncallback into the model to query the model state. The result is again inconsistent lock ordering, with the attendant risk of deadlock. Inhisweblog,[1]SunVPGrahamHamiltonnicelysumsupthechallenges,describingwhythemultithreadedGUItoolkitis oneoftherecurring"faileddreams"ofcomputerscience. [1]http://weblogs.java.net/blog/kgh/archive/2004/10 IbelieveyoucanprogramsuccessfullywithmultithreadedGUItoolkitsifthetoolkitisverycarefullydesigned;ifthe toolkit exposes its locking methodology in gory detail; if you are very smart, very careful, and have a global understandingofthewholestructureofthetoolkit.Ifyougetoneofthesethingsslightlywrong,thingswillmostly work,butyouwillgetoccasionalhangs(duetodeadlocks)orglitches(duetoraces).Thismultithreadedapproachworks bestforpeoplewhohavebeenintimatelyinvolvedinthedesignofthetoolkit. Unfortunately,Idon'tthinkthissetofcharacteristicsscalestowidespreadcommercialuse.Whatyoutendtoendup withisnormalsmartprogrammersbuildingappsthatdon'tquiteworkreliablyforreasonsthatarenotatallobvious.So theauthorsgetverydisgruntledandfrustratedandusebadwordsonthepoorinnocenttoolkit.  118 JavaConcurrencyInPractice SingleͲthreaded GUI frameworks achieve thread safety via thread confinement; all GUI objects, including visual componentsanddatamodels,areaccessedexclusivelyfromtheeventthread.Ofcourse,thisjustpushessomeofthe threadsafetyburdenbackontotheapplicationdeveloper,whomustmakesuretheseobjectsareproperlyconfined. 9.1.1.SequentialEventProcessing GUI applications are oriented around processing fineͲgrained events such as mouse clicks, key presses, or timer expirations.Eventsareakindoftask;theeventhandlingmachineryprovidedbyAWTandSwingisstructurallysimilarto anExecutor. BecausethereisonlyasinglethreadforprocessingGUItasks,theyareprocessedsequentiallyͲonetaskfinishesbefore thenextonebegins,andnotwotasksoverlap.KnowingthismakeswritingtaskcodeeasierͲyoudon'thavetoworry aboutinterferencefromothertasks. Thedownsideofsequentialtaskprocessingisthatifonetasktakesalongtimetoexecute,othertasksmustwaituntilit isfinished.Ifthoseothertasksareresponsibleforrespondingtouserinputorprovidingvisualfeedback,theapplication willappeartohavefrozen.Ifalengthytaskisrunningintheeventthread,theusercannotevenclick"Cancel"because the cancel button listener is not called until the lengthy task completes. Therefore, tasks that execute in the event threadmustreturncontroltotheeventthreadquickly.ToinitiatealongrunningtasksuchasspellͲcheckingalarge document,searchingthefilesystem,orfetchingaresourceoveranetwork,youmustrunthattaskinanotherthreadso controlcanreturnquicklytotheeventthread.ToupdateaprogressindicatorwhilealongͲrunningtaskexecutesor provide visual feedback when it completes, you again need to execute code in the event thread. This can get complicatedquickly. 9.1.2.ThreadConfinementinSwing AllSwingcomponents(suchasJButtonandJTable)anddatamodelobjects(suchasTableModelandTReeModel)are confinedtotheeventthread,soanycodethataccessestheseobjectsmustrunintheeventthread.GUIobjectsarekept consistentnotbysynchronization,butbythreadconfinement.Theupsideisthattasksthatrunintheeventthreadneed not worry about synchronization when accessing presentation objects; the downside is that you cannot access presentationobjectsfromoutsidetheeventthreadatall. TheSwingsingleͲthreadrule:Swingcomponentsandmodelsshouldbecreated,modified,andqueriedonlyfromthe eventͲdispatchingthread. Aswithallrules,thereareafewexceptions.AsmallnumberofSwingmethodsmaybecalledsafelyfromanythread; theseareclearlyidentifiedintheJavadocasbeingthreadͲsafe.OtherexceptionstothesingleͲthreadruleinclude: x SwingUtilities.isEventDispatchThread,whichdetermineswhetherthecurrentthreadistheeventthread; x SwingUtilities.invokeLater,whichschedulesaRunnableforexecutionontheeventthread(callablefrom anythread); x SwingUtilities.invokeAndWait, which schedules a Runnable task for execution on the event thread and blocksthecurrentthreaduntilitcompletes(callableonlyfromanonͲGUIthread); x methodstoenqueuearepaintorrevalidationrequestontheeventqueue(callablefromanythread);and x methodsforaddingandremovinglisteners(canbecalledfromanythread,butlistenerswillalwaysbeinvoked intheeventthread). TheinvokeLaterand invokeAndWaitmethodsfunctionalotlikeanExecutor.Infact,itistrivialtoimplementthe threadingͲrelatedmethodsfromSwingUtilitiesusingasingleͲthreadedExecutor,asshowninListing9.1.Thisisnot how SwingUtilities is actually implemented, as Swing predates the Executor framework, but is probably how it wouldbeifSwingwerebeingimplementedtoday. TheSwingeventthreadcanbethoughtofasasingleͲthreadedExecutorthatprocessestasksfromtheeventqueue.As withthreadpools,sometimestheworkerthreaddiesandisreplacedbyanewone,butthisshouldbetransparentto tasks. Sequential, singleͲthreaded execution is a sensible execution policy when tasks are shortͲlived, scheduling predictabilityisnotimportant,oritisimperativethattasksnotexecuteconcurrently. GuiExecutor in Listing 9.2 is an Executor that delegates tasks to SwingUtilities for execution. It could be implementedintermsofotherGUIframeworksaswell;forexample,SWTprovidestheDisplay.asyncExecmethod, whichissimilartoSwing'sinvokeLater.  1195BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications 9.2.ShortǦrunningGUITasks InaGUIapplication,eventsoriginateintheeventthreadandbubbleuptoapplicationͲprovidedlisteners,whichwill probablyperformsomecomputationthataffectsthepresentationobjects.Forsimple,shortͲrunningtasks,theentire actioncanstayintheeventthread;forlongerͲrunningtasks,someoftheprocessingshouldbeoffloadedtoanother thread. Inthesimplecase,confiningpresentationobjectstotheeventthreadiscompletelynatural.Listing9.3createsabutton whosecolorchangesrandomlywhenpressed.Whentheuserclicksonthebutton,thetoolkitdeliversanActionEvent intheeventthreadtoallregisteredactionlisteners.Inresponse,theactionlistenerpicksanewcolorandchangesthe button's background color. So the event originates in the GUI toolkit and is delivered to the application, and the applicationmodifiestheGUIinresponsetotheuser'saction.Controlneverhastoleavetheeventthread,asillustrated inFigure9.1. Figure9.1.ControlFlowofaSimpleButtonClick. ThistrivialexamplecharacterizesthemajorityofinteractionsbetweenGUIapplicationsandGUItoolkits.Solongas tasksareshortͲlivedandaccessonlyGUIobjects(orotherthreadͲconfinedorthreadͲsafeapplicationobjects),youcan almosttotallyignorethreadingconcernsanddoeverythingfromtheeventthread,andtherightthinghappens.   120 JavaConcurrencyInPractice Listing9.1.ImplementingSwingUtilitiesUsinganExecutor. public class SwingUtilities { private static final ExecutorService exec = Executors.newSingleThreadExecutor(new SwingThreadFactory()); private static volatile Thread swingThread; private static class SwingThreadFactory implements ThreadFactory { public Thread newThread(Runnable r) { swingThread = new Thread(r); return swingThread; } } public static boolean isEventDispatchThread() { return Thread.currentThread() == swingThread; } public static void invokeLater(Runnable task) { exec.execute(task); } public static void invokeAndWait(Runnable task) throws InterruptedException, InvocationTargetException { Future f = exec.submit(task); try { f.get(); } catch (ExecutionException e) { throw new InvocationTargetException(e); } } } Listing9.2.ExecutorBuiltAtopSwingUtilities. public class GuiExecutor extends AbstractExecutorService { // Singletons have a private constructor and a public factory private static final GuiExecutor instance = new GuiExecutor(); private GuiExecutor() { } public static GuiExecutor instance() { return instance; } public void execute(Runnable r) { if (SwingUtilities.isEventDispatchThread()) r.run(); else SwingUtilities.invokeLater(r); } // Plus trivial implementations of lifecycle methods } Listing9.3.SimpleEventListener. final Random random = new Random(); final JButton button = new JButton("Change Color"); ... button.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { button.setBackground(new Color(random.nextInt())); } }); Aslightlymorecomplicatedversionofthissamescenario,illustratedinFigure9.2,involvestheuseofaformaldata modelsuchasaTableModelortreeModel.Swingsplitsmostvisualcomponentsintotwoobjects,amodelandaview. Thedatatobedisplayedresidesinthemodelandtherulesgoverninghowitisdisplayedresideintheview.Themodel objectscanfireeventsindicatingthatthemodeldatahaschanged,andviewssubscribetotheseevents.Whentheview receivesaneventindicatingthemodeldatamayhavechanged,itqueriesthemodelforthenewdataandupdatesthe display.Soinabuttonlistenerthatmodifiesthecontentsofatable,theactionlistenerwouldupdatethemodelandcall oneofthe fireXxxmethods,whichwouldinturninvoketheview'stablemodellisteners,whichwouldupdatethe view.Again,controlneverleavestheeventthread.(TheSwingdatamodel fireXxxmethodsalwayscallthemodel listenersdirectlyratherthansubmittinganeweventtotheeventqueue,sothefireXxxmethodsmustbecalledonly fromtheeventthread.)  1215BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications Figure9.2.ControlFlowwithSeparateModelandViewObjects. 9.3.LongǦrunningGUITasks IfalltaskswereshortͲrunning(andtheapplicationhadnosignificantnonͲGUIportion),thentheentireapplicationcould runwithintheeventthreadandyouwouldn'thavetopayanyattentiontothreadsatall.However,sophisticatedGUI applicationsmayexecutetasksthatmaytakelongerthantheuseriswillingtowait,suchasspellchecking,background compilation,orfetchingremoteresources.ThesetasksmustruninanotherthreadsothattheGUIremainsresponsive whiletheyrun. Swingmakesiteasytohaveataskrunintheeventthread,but(priortoJava6)doesn'tprovideanymechanismfor helpingGUItasksexecutecodeinotherthreads.Butwedon'tneedSwingtohelpushere:wecancreateourown ExecutorforprocessinglongͲrunningtasks.AcachedthreadpoolisagoodchoiceforlongͲrunningtasks;onlyrarelydo GUIapplicationsinitiatealargenumberoflongͲrunningtasks,sothereislittleriskofthepoolgrowingwithoutbound. WestartwithasimpletaskthatdoesnotsupportcancellationorprogressindicationandthatdoesnotupdatetheGUI on completion, and then add those features one by one. Listing 9.4 shows an action listener, bound to a visual component,thatsubmitsalongͲrunningtasktoanExecutor.Despitethetwolayersofinnerclasses,havingaGUItask initiateataskinthismannerisfairlystraightforward:theUIactionlisteneriscalledintheeventthreadandsubmitsa Runnabletoexecuteinthethreadpool. ThisexamplegetsthelongͲrunningtaskoutoftheeventthreadina"fireandforget"manner,whichisprobablynot veryuseful.ThereisusuallysomesortofvisualfeedbackwhenalongͲrunningtaskcompletes.Butyoucannotaccess presentationobjectsfromthebackgroundthread,sooncompletionthetaskmustsubmitanothertasktoruninthe eventthreadtoupdatetheuserinterface. Listing9.4.BindingaLongǦrunningTasktoaVisualComponent. ExecutorService backgroundExec = Executors.newCachedThreadPool(); ... button.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { backgroundExec.execute(new Runnable() { public void run() { doBigComputation(); } }); }}); Listing9.5illustratestheobviouswaytodothis,whichisstartingtogetcomplicated;we'renowuptothreelayersof innerclasses.Theactionlistenerfirstdimsthebuttonandsetsalabelindicatingthatacomputationisinprogress,then submitsatasktothebackgroundexecutor.Whenthattaskfinishes,itqueuesanothertasktorunintheeventthread, whichreenablesthebuttonandrestoresthelabeltext.  122 JavaConcurrencyInPractice Listing9.5.LongǦrunningTaskwithUserFeedback. button.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { button.setEnabled(false); label.setText("busy"); backgroundExec.execute(new Runnable() { public void run() { try { doBigComputation(); } finally { GuiExecutor.instance().execute(new Runnable() { public void run() { button.setEnabled(true); label.setText("idle"); } }); } } }); } }); Thetasktriggeredwhenthebuttonispressediscomposedofthreesequentialsubtaskswhoseexecutionalternates betweentheeventthreadandthebackgroundthread.ThefirstsubtaskupdatestheuserinterfacetoshowthatalongͲ running operation has begun and starts the second subtask in a background thread. Upon completion, the second subtaskqueuesthethirdsubtasktorunagainintheeventthread,whichupdatestheuserinterfacetoreflectthatthe operationhascompleted.Thissortof"threadhopping"istypicalofhandlinglongͲrunningtasksinGUIapplications. 9.3.1.Cancellation Anytaskthattakeslongenoughtoruninanotherthreadprobablyalsotakeslongenoughthattheusermightwantto cancelit.Youcouldimplementcancellationdirectlyusingthreadinterruption,butitismucheasiertouseFuture,which wasdesignedtomanagecancellabletasks. WhenyoucallcancelonaFuturewithmayInterruptIfRunningsettotrue,theFutureimplementationinterrupts thethreadthatisexecutingthetaskifitiscurrentlyrunning.Ifyourtaskiswrittentoberesponsivetointerruption,it canreturnearlyifitiscancelled.Listing9.6illustratesataskthatpollsthethread'sinterruptedstatusandreturnsearly oninterruption. Listing9.6.CancellingaLongǦrunningTask. Future runningTask = null; // thread-confined ... startButton.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { if (runningTask != null) { runningTask = backgroundExec.submit(new Runnable() { public void run() { while (moreWork()) { if (Thread.currentThread().isInterrupted()) { cleanUpPartialWork(); break; } doSomeWork(); } } }); }; }}); cancelButton.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent event) { if (runningTask != null) runningTask.cancel(true); }}); BecauserunningTaskisconfinedtotheeventthread,nosynchronizationisrequiredwhensettingorcheckingit,and thestartbuttonlistenerensuresthatonlyonebackgroundtaskisrunningatatime.However,itwouldbebettertobe notifiedwhenthetaskcompletessothat,forexample,thecancelbuttoncouldbedisabled.Weaddressthisinthenext section. 9.3.2.ProgressandCompletionIndication UsingaFuturetorepresentalongͲrunningtaskgreatlysimplifiedimplementingcancellation.FutureTaskalsohasa donehookthatsimilarlyfacilitatescompletionnotification.AfterthebackgroundCallablecompletes,doneiscalled.By  1235BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications having done TRigger a completion task in the event thread, we can construct a BackgroundTask class providing an onCompletionhookthatiscalledintheeventthread,asshowninListing9.7. BackgroundTaskalsosupportsprogressindication.ThecomputemethodcancallsetProgress,indicatingprogressin numericalterms.ThiscausesonProgresstobecalledfromtheeventthread,whichcanupdatetheuserinterfaceto indicateprogressvisually. ToimplementaBackgroundTaskyouneedonlyimplementcompute,whichiscalledinthebackgroundthread.Youalso havetheoptionofoverridingonCompletionandonProgress,whichareinvokedintheeventthread. BasingBackgroundTaskonFutureTaskalsosimplifiescancellation.Ratherthanhavingtopollthethread'sinterrupted status,computecancallFuture. is-Cancelled.Listing9.8recaststheexamplefromListing9.6using Background- Task. 9.3.3.SwingWorker We'vebuiltasimpleframeworkusingFutureTaskandExecutortoexecutelongrunningtasksinbackgroundthreads without undermining the responsiveness of the GUI. These techniques can be applied to any singleͲthreaded GUI framework, not just Swing. In Swing, many of the features developed here are provided by the SwingWorker class, including cancellation, completion notification, and progress indication. Various versions of SwingWorker have been publishedinTheSwingConnectionandTheJavaTutorial,andanupdatedversionisincludedinJava6. 9.4.SharedDataModels Swingpresentationobjects,includingdatamodelobjectssuchasTableModelortreeModel,areconfinedtotheevent thread.InsimpleGUIprograms,allthemutablestateisheldinthepresentationobjectsandtheonlythreadbesidesthe eventthreadisthemainthread.IntheseprogramsenforcingthesingleͲthreadruleiseasy:don'taccessthedatamodel orpresentationcomponentsfromthemainthread.Morecomplicatedprogramsmayuseotherthreadstomovedatato orfromapersistentstore,suchasafilesystemordatabase,soasnottocompromiseresponsiveness. Inthesimplestcase,thedatainthedatamodelisenteredbytheuserorloadedstaticallyfromafileorotherdata sourceatapplicationstartup,inwhichcasethedataisnevertouchedbyanythreadotherthantheeventthread.But sometimesthepresentationmodelobjectisonlyaviewontoanotherdatasource,suchasadatabase,filesystem,or remoteservice.Inthiscase,morethanonethreadislikelytotouchthedataasitgoesintooroutoftheapplication.  124 JavaConcurrencyInPractice Listing9.7.BackgroundTaskClassSupportingCancellation,CompletionNotification,andProgressNotification. abstract class BackgroundTask implements Runnable, Future { private final FutureTask computation = new Computation(); private class Computation extends FutureTask { public Computation() { super(new Callable() { public V call() throws Exception { return BackgroundTask.this.compute() ; } }); } protected final void done() { GuiExecutor.instance().execute(new Runnable() { public void run() { V value = null; Throwable thrown = null; boolean cancelled = false; try { value = get(); } catch (ExecutionException e) { thrown = e.getCause(); } catch (CancellationException e) { cancelled = true; } catch (InterruptedException consumed) { } finally { onCompletion(value, thrown, cancelled); } }; }); } } protected void setProgress(final int current, final int max) { GuiExecutor.instance().execute(new Runnable() { public void run() { onProgress(current, max); } }); } // Called in the background thread protected abstract V compute() throws Exception; // Called in the event thread protected void onCompletion(V result, Throwable exception, boolean cancelled) { } protected void onProgress(int current, int max) { } // Other Future methods forwarded to computation } Listing9.8.InitiatingaLongǦrunning,CancellableTaskwithBackgroundTask. public void runInBackground(final Runnable task) { startButton.addActionListener(new ActionListener() { public void actionPerformed(ActionEvent e) { class CancelListener implements ActionListener { BackgroundTask task; public void actionPerformed(ActionEvent event) { if (task != null) task.cancel(true); } } final CancelListener listener = new CancelListener(); listener.task = new BackgroundTask() { public Void compute() { while (moreWork() && !isCancelled()) doSomeWork(); return null; } public void onCompletion(boolean cancelled, String s, Throwable exception) { cancelButton.removeActionListener(listener); label.setText("done"); } }; cancelButton.addActionListener(listener); backgroundExec.execute(task); } }); } For example, you might display the contents of a remote file system using a tree control. You wouldn't want to enumeratetheentirefilesystembeforeyoucandisplaythetreecontrolthatwouldtaketoomuchtimeandmemory. Instead, the tree can be lazily populated as nodes are expanded. Enumerating even a single directory on a remote volumecantakealongtime,soyoumaywanttodotheenumerationinabackgroundtask.Whenthebackgroundtask  1255BPartII:StructuringConcurrentApplications Ͳ 21BChapter9.GUIApplications completes,youhavetogetthedataintothetreemodelsomehow.ThiscouldbedonebyusingathreadͲsafetree model,by"pushing"thedatafromthebackgroundtasktotheeventthreadbypostingataskwithinvokeLater,orby havingtheeventthreadpolltoseeifthedataisavailable. 9.4.1.ThreadǦsafeDataModels Aslongasresponsivenessisnotundulyaffectedbyblocking,theproblemofmultiplethreadsoperatingonthedatacan beaddressedwithathreadͲsafedatamodel.IfthedatamodelsupportsfineͲgrainedconcurrency,theeventthreadand background threads should be able to share it without responsiveness problems. For example, DelegatingVehicleTracker on page 65 uses an underlying ConcurrentHashMap whose retrieval operations offer a highdegreeofconcurrency.Thedownsideisthatitdoesnotofferaconsistentsnapshotofthedata,whichmayormay notbearequirement.ThreadͲsafedatamodelsmustalsogenerateeventswhenthemodelhasbeenupdated,sothat viewscanbeupdatedwhenthedatachanges. Itmaysometimesbepossibletogetthreadsafety,consistencyandgoodresponsivenesswithaversioneddatamodel suchasCopyOnWriteArrayList[CPJ2.2.3.3].WhenyouacquireaniteratorforacopyͲonͲwritecollection,thatiterator traverses the collection as it existed when the iterator was created. However, copyͲonͲwrite collections offer good performanceonlywhentraversalsgreatlyoutnumbermodifications,whichwouldprobablynotbethecasein,say,a vehicle tracking application. More specialized versioned data structures may avoid this restriction, but building versioneddatastructuresthatprovidebothefficientconcurrentaccessanddonotretainoldversionsofdatalonger thanneededisnoteasy,andthusshouldbeconsideredonlywhenotherapproachesarenotpractical. 9.4.2.SplitDataModels From the perspective of the GUI, the Swing table model classes like TableModel and treeModel are the official repository for data to be displayed. However, these model objects are often themselves "views" of other objects managedbytheapplication.AprogramthathasbothapresentationͲdomainandanapplicationdomaindatamodelis saidtohaveasplitͲmodeldesign(Fowler,2005). InasplitͲmodeldesign,thepresentationmodelisconfinedtotheeventthreadandtheothermodel,thesharedmodel, isthreadͲsafeandmaybeaccessedbyboththeeventthreadandapplicationthreads.Thepresentationmodelregisters listenerswiththesharedmodelsoitcanbenotifiedofupdates.Thepresentationmodelcanthenbeupdatedfromthe sharedmodelbyembeddingasnapshotoftherelevantstateintheupdatemessageorbyhavingthepresentation modelretrievethedatadirectlyfromthesharedmodelwhenitreceivesanupdateevent. Thesnapshotapproachissimple,buthaslimitations.Itworkswellwhenthedatamodelissmall,updatesarenottoo frequent,andthestructureofthetwomodelsissimilar.Ifthedatamodelislargeorupdatesareveryfrequent,orifone or both sides of the split contain information that is not visible to the other side, it can be more efficient to send incremental updates instead of entire snapshots. This approach has the effect of serializing updates on the shared model and recreating them in the event thread against the presentation model. Another advantage of incremental updatesisthatfinerͲgrainedinformationaboutwhatchangedcanimprovetheperceivedqualityofthedisplayifonly onevehiclemoves,wedon'thavetorepainttheentiredisplay,justtheaffectedregions. ConsiderasplitͲmodeldesignwhenadatamodelmustbesharedbymorethanonethreadandimplementingathreadͲ safedatamodelwouldbeinadvisablebecauseofblocking,consistency,orcomplexityreasons. 9.5.OtherFormsofSingleǦthreadedSubsystems ThreadconfinementisnotrestrictedtoGUIs:itcanbeusedwheneverafacilityisimplementedasasingleͲthreaded subsystem.Sometimesthreadconfinementisforcedonthedeveloperforreasonsthathavenothingtodowithavoiding synchronizationordeadlock.Forexample,somenativelibrariesrequirethatallaccesstothelibrary,evenloadingthe librarywithSystem.loadLibrary,bemadefromthesamethread. BorrowingfromtheapproachtakenbyGUIframeworks,youcaneasilycreateadedicatedthreadorsingleͲthreaded executorforaccessingthenativelibrary,andprovideaproxyobjectthatinterceptscallstothethreadͲconfinedobject andsubmitsthemastaskstothededicatedthread.FutureandnewSingleThreadExecutorworktogethertomakethis easy;theproxymethodcansubmitthetaskandimmediatelycallFuture.gettowaitfortheresult.(Iftheclasstobe threadͲconfinedimplementsaninterface,youcanautomatetheprocessofhavingeachmethodsubmitaCallabletoa backgroundthreadexecutorandwaitingfortheresultusingdynamicproxies.)  126 JavaConcurrencyInPractice Summary GUIframeworksarenearlyalwaysimplementedassingleͲthreadedsubsystemsinwhichallpresentationͲrelatedcode runs as tasks in an event thread. Because there is only a single event thread, longͲrunning tasks can compromise responsiveness and so should be executed in background threads. Helper classes like SwingWorker or the BackgroundTaskclassbuilthere,whichprovidesupportforcancellation,progressindication,andcompletionindication, cansimplifythedevelopmentoflongͲrunningtasksthathavebothGUIandnonͲGUIcomponents.  1276BPartIII:Liveness,Performance,andTesting Ͳ 21BChapter9.GUIApplications PartIII:Liveness,Performance,andTesting  Chapter10.AvoidingLivenessHazards Chapter11.PerformanceandScalability Chapter12.TestingConcurrentPrograms  128 JavaConcurrencyInPractice Chapter10.AvoidingLivenessHazards Thereisoftenatensionbetweensafetyandliveness.Weuselockingtoensurethreadsafety,butindiscriminateuseof locking can cause lockͲordering deadlocks. Similarly, we use thread pools and semaphores to bound resource consumption,butfailuretounderstandtheactivitiesbeingboundedcancauseresourcedeadlocks.Javaapplicationsdo notrecoverfromdeadlock,soitisworthwhiletoensurethatyourdesignprecludestheconditionsthatcouldcauseit. Thischapterexploressomeofthecausesoflivenessfailuresandwhatcanbedonetopreventthem. 10.1.Deadlock Deadlockisillustratedbytheclassic,ifsomewhatunsanitary,"diningphilosophers"problem.Fivephilosophersgoout forChinesefoodandareseatedatacirculartable.Therearefivechopsticks(notfivepairs),oneplacedbetweeneach pairofdiners.Thephilosophersalternatebetweenthinkingandeating.Eachneedstoacquiretwochopsticksforlong enoughtoeat,butcanthenputthechopsticksbackandreturntothinking.TherearesomechopstickͲmanagement algorithms that let everyone eat on a more or less timely basis (a hungry philosopher tries to grab both adjacent chopsticks,butifoneisnotavailable,putsdowntheonethatisavailableandwaitsaminuteorsobeforetryingagain), andsomethatcanresultinsomeorallofthephilosophersdyingofhunger(eachphilosopherimmediatelygrabsthe chopsticktohisleftandwaitsforthechopsticktohisrighttobeavailablebeforeputtingdowntheleft).Thelatter situation,whereeachhasaresourceneededbyanotherandiswaitingforaresourceheldbyanother,andwillnot releasetheonetheyholduntiltheyacquiretheonetheydon't,illustratesdeadlock. Whenathreadholdsalockforever,otherthreadsattemptingtoacquirethatlockwillblockforeverwaiting.When threadAholdslockLandtriestoacquirelockM,butatthesametimethreadBholdsMandtriestoacquireL,both threadswillwaitforever.Thissituationisthesimplestcaseofdeadlock(ordeadlyembrace),wheremultiplethreads waitforeverduetoacycliclockingdependency.(Thinkofthethreadsasthenodesofadirectedgraphwhoseedges representtherelation"ThreadAiswaitingforaresourceheldbythreadB".Ifthisgraphiscyclical,thereisadeadlock.) Databasesystemsaredesignedtodetectandrecoverfromdeadlock.Atransactionmayacquiremanylocks,andlocks are held until the transaction commits. So it is quite possible, and in fact not uncommon, for two transactions to deadlock.Withoutintervention,theywouldwaitforever(holdinglocksthatareprobablyrequiredbyothertransactions aswell).Butthedatabaseserverisnotgoingtoletthishappen.Whenitdetectsthatasetoftransactionsisdeadlocked (whichitdoesbysearchingtheisͲwaitingͲforgraphforcycles),itpicksavictimandabortsthattransaction.Thisreleases thelocksheldbythevictim,allowingtheothertransactionstoproceed.Theapplicationcanthenretrytheaborted transaction,whichmaybeabletocompletenowthatanycompetingtransactionshavecompleted. TheJVMisnotnearlyashelpfulinresolvingdeadlocksasdatabaseserversare.WhenasetofJavathreadsdeadlock, that'stheendofthegamethosethreadsarepermanentlyoutofcommission.Dependingonwhatthosethreadsdo,the application may stall completely, or a particular subsystem may stall, or performance may suffer. The only way to restoretheapplicationtohealthistoabortandrestartitandhopethesamethingdoesn'thappenagain. Likemanyotherconcurrencyhazards,deadlocksrarelymanifestthemselvesimmediately.Thefactthataclasshasa potentialdeadlockdoesn'tmeanthatiteverwilldeadlock,justthatitcan.Whendeadlocksdomanifestthemselves,itis oftenattheworstpossibletimeunderheavyproductionload. 10.1.1.LockǦorderingDeadlocks LeftRightDeadlockinListing10.1isatriskfordeadlock.The leftRightand rightLeftmethodseachacquirethe leftandrightlocks.IfonethreadcallsleftRightandanothercallsrightLeft,andtheiractionsareinterleavedas showninFigure10.1,theywilldeadlock. Figure10.1.UnluckyTiminginLeftRightDeadlock.    1296BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards ThedeadlockinLeftRightDeadlockcameaboutbecausethetwothreadsattemptedtoacquirethesamelocksina differentorder.Iftheyaskedforthelocksinthesameorder,therewouldbenocycliclockingdependencyandtherefore nodeadlock.IfyoucanguaranteethateverythreadthatneedslocksLandMatthesametimealwaysacquiresLandM inthesameorder,therewillbenodeadlock. AprogramwillbefreeoflockͲorderingdeadlocksifallthreadsacquirethelockstheyneedinafixedglobalorder. Verifyingconsistentlockorderingrequiresaglobalanalysisofyourprogram'slockingbehavior.Itisnotsufficientto inspectcodepathsthatacquiremultiplelocksindividually;bothleftRightandrightLeftare"reasonable"waysto acquirethetwolocks,theyarejustnotcompatible.Whenitcomestolocking,thelefthandneedstoknowwhatthe righthandisdoing. Listing10.1.SimpleLockǦorderingDeadlock.Don'tDothis. // Warning: deadlock-prone! public class LeftRightDeadlock { private final Object left = new Object(); private final Object right = new Object(); public void leftRight() { synchronized (left) { synchronized (right) { doSomething(); } } } public void rightLeft() { synchronized (right) { synchronized (left) { doSomethingElse(); } } } } 10.1.2.DynamicLockOrderDeadlocks Sometimes it is not obvious that you have sufficient control over lock ordering to prevent deadlocks. Consider the harmlessͲlookingcodeinListing10.2thattransfersfundsfromoneaccounttoanother.Itacquiresthelocksonboth Accountobjectsbeforeexecutingthetransfer,ensuringthatthebalancesareupdatedatomicallyandwithoutviolating invariantssuchas"anaccountcannothaveanegativebalance". HowcanTRansferMoneydeadlock?Itmayappearasifallthethreadsacquiretheirlocksinthesameorder,butinfact the lock order depends on the order of arguments passed to transferMoney, and these in turn might depend on externalinputs.DeadlockcanoccuriftwothreadscalltransferMoneyatthesametime,onetransferringfromXtoY, andtheotherdoingtheopposite: Listing10.2.DynamicLockǦorderingDeadlock.Don'tDothis. // Warning: deadlock-prone! public void transferMoney(Account fromAccount, Account toAccount, DollarAmount amount) throws InsufficientFundsException { synchronized (fromAccount) { synchronized (toAccount) { if (fromAccount.getBalance().compareTo(amount) < 0) throw new InsufficientFundsException(); else { fromAccount.debit(amount); toAccount.credit(amount); } } } }   130 JavaConcurrencyInPractice A: transferMoney(myAccount, yourAccount, 10); B: transferMoney(yourAccount, myAccount, 20); Withunluckytiming,AwillacquirethelockonmyAccountandwaitforthelockonyourAccount,whileBisholdingthe lockonyourAccountandwaitingforthelockonmyAccount. DeadlockslikethisonecanbespottedthesamewayasinListing10.1lookfornestedlockacquisitions.Sincetheorder of arguments is out of our control, to fix the problem we must induce an ordering on the locks and acquire them accordingtotheinducedorderingconsistentlythroughouttheapplication. OnewaytoinduceanorderingonobjectsistouseSystem.identityHashCode,whichreturnsthevaluethatwouldbe returnedbyObject.hashCode.Listing10.3showsaversionoftransferMoneythatusesSystem.identityHashCodeto inducealockordering.Itinvolvesafewextralinesofcode,buteliminatesthepossibilityofdeadlock. In the rare case that two objects have the same hash code, we must use an arbitrary means of ordering the lock acquisitions,andthisreintroducesthepossibilityofdeadlock.Topreventinconsistentlockorderinginthiscase,athird "tiebreaking"lockisused.ByacquiringthetieͲbreakinglockbeforeacquiringeitherAccountlock,weensurethatonly onethreadatatimeperformstheriskytaskofacquiringtwolocksinanarbitraryorder,eliminatingthepossibilityof deadlock(solongasthismechanismisusedconsistently).Ifhashcollisionswerecommon,thistechniquemightbecome a concurrency bottleneck (just as having a single, programͲwide lock would), but because hash collisions with System.identityHashCodearevanishinglyinfrequent,thistechniqueprovidesthatlastbitofsafetyatlittlecost. Listing10.3.InducingaLockOrderingtoAvoidDeadlock. private static final Object tieLock = new Object(); public void transferMoney(final Account fromAcct, final Account toAcct, final DollarAmount amount) throws InsufficientFundsException { class Helper { public void transfer() throws InsufficientFundsException { if (fromAcct.getBalance().compareTo(amount) < 0) throw new InsufficientFundsException(); else { fromAcct.debit(amount); toAcct.credit(amount); } } } int fromHash = System.identityHashCode(fromAcct); int toHash = System.identityHashCode(toAcct); if (fromHash < toHash) { synchronized (fromAcct) { synchronized (toAcct) { new Helper().transfer(); } } } else if (fromHash > toHash) { synchronized (toAcct) { synchronized (fromAcct) { new Helper().transfer(); } } } else { synchronized (tieLock) { synchronized (fromAcct) { synchronized (toAcct) { new Helper().transfer(); } } } } } IfAccounthasaunique,immutable,comparablekeysuchasanaccountnumber,inducingalockorderingiseveneasier: orderobjectsbytheirkey,thuseliminatingtheneedforthetieͲbreakinglock. Youmaythinkwe'reoverstatingtheriskofdeadlockbecauselocksareusuallyheldonlybriefly,butdeadlocksarea seriousprobleminrealsystems.AproductionapplicationmayperformbillionsoflockacquireͲreleasecyclesperday. Onlyoneofthoseneedstobetimedjustwrongtobringtheapplicationtodeadlock,andevenathoroughloadͲtesting regimenmaynotdisclosealllatentdeadlocks.[1]DemonstrateDeadlockinListing10.4[2]deadlocksfairlyquicklyonmost systems.  1316BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards [1]Ironically,holdinglocksforshortperiodsoftime,asyouaresupposedtodotoreducelockcontention,increasesthelikelihoodthattestingwill notdiscloselatentdeadlockrisks. [2]Forsimplicity,DemonstrateDeadlockignorestheissueofnegativeaccountbalances. Listing10.4.DriverLoopthatInducesDeadlockUnderTypicalConditions. public class DemonstrateDeadlock { private static final int NUM_THREADS = 20; private static final int NUM_ACCOUNTS = 5; private static final int NUM_ITERATIONS = 1000000; public static void main(String[] args) { final Random rnd = new Random(); final Account[] accounts = new Account[NUM_ACCOUNTS]; for (int i = 0; i < accounts.length; i++) accounts[i] = new Account(); class TransferThread extends Thread { public void run() { for (int i=0; i taxis; @GuardedBy("this") private final Set availableTaxis; public Dispatcher() { taxis = new HashSet(); availableTaxis = new HashSet(); } public synchronized void notifyAvailable(Taxi taxi) { availableTaxis.add(taxi); } public synchronized Image getImage() { Image image = new Image(); for (Taxi t : taxis) image.drawMarker(t.getLocation()); return image; } } TaxiandDispatcherinListing10.5canbeeasilyrefactoredtouseopencallsandthuseliminatethedeadlockrisk.This involvesshrinkingthesynchronizedblockstoguardonlyoperationsthatinvolvesharedstate,asinListing10.6.Very often, the cause of problems like those in Listing 10.5 is the use of synchronized methods instead of smaller synchronized blocks for reasons of compact syntax or simplicity rather than because the entire method must be guarded byalock. (Asa bonus,shrinking the synchronizedblockmayalsoimprovescalabilityaswell;seeSection 11.4.1foradviceonsizingsynchronizedblocks.) Strive to use open calls throughout your program. Programs that rely on open calls are far easier to analyze for deadlockͲfreedomthanthosethatallowcallstoalienmethodswithlocksheld. Restructuringasynchronizedblocktoallowopencallscansometimeshaveundesirableconsequences,sinceittakesan operationthatwasatomicandmakesitnotatomic.Inmanycases,thelossofatomicityisperfectlyacceptable;there's noreasonthatupdatingataxi'slocationandnotifyingthedispatcherthatitisreadyforanewdestinationneedbean atomicoperation.Inothercases,thelossofatomicityisnoticeablebutthesemanticchangesarestillacceptable.Inthe deadlockͲproneversion,getImageproducesacompletesnapshotofthefleetlocationsatthatinstant;intherefactored version,itfetchesthelocationofeachtaxiatslightlydifferenttimes. Insomecases,however,thelossofatomicityisaproblem,andhereyouwillhavetouseanothertechniquetoachieve atomicity.Onesuchtechniqueistostructureaconcurrentobjectsothatonlyonethreadcanexecutethecodepath followingtheopencall.Forexample,whenshuttingdownaservice,youmaywanttowaitforinͲprogressoperationsto complete and then release resources used by the service. Holding the service lock while waiting for operations to completeisinherentlydeadlockͲprone,butreleasingtheservicelockbeforetheserviceisshutdownmayletother threads start new operations. The solution is to hold the lock long enough to update the service state to "shutting  1336BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards down"sothatotherthreadswantingtostartnewoperationsincludingshuttingdowntheserviceseethattheserviceis unavailable,anddonottry.Youcanthenwaitforshutdowntocomplete,knowingthatonlytheshutdownthreadhas accesstotheservicestateaftertheopencallcompletes.Thus,ratherthanusinglockingtokeeptheotherthreadsout ofacriticalsectionofcode,thistechniquereliesonconstructingprotocolssothatotherthreadsdon'ttrytogetin. 10.1.5.ResourceDeadlocks Justasthreadscandeadlockwhentheyareeachwaitingforalockthattheotherholdsandwillnotrelease,theycan alsodeadlockwhenwaitingforresources. Listing10.6.UsingOpenCallstoAvoidingDeadlockBetweenCooperatingObjects. @ThreadSafe class Taxi { @GuardedBy("this") private Point location, destination; private final Dispatcher dispatcher; ... public synchronized Point getLocation() { return location; } public synchronized void setLocation(Point location) { boolean reachedDestination; synchronized (this) { this.location = location; reachedDestination = location.equals(destination); } if (reachedDestination) dispatcher.notifyAvailable(this); } } @ThreadSafe class Dispatcher { @GuardedBy("this") private final Set taxis; @GuardedBy("this") private final Set availableTaxis; ... public synchronized void notifyAvailable(Taxi taxi) { availableTaxis.add(taxi); } public Image getImage() { Set copy; synchronized (this) { copy = new HashSet(taxis); } Image image = new Image(); for (Taxi t : copy) image.drawMarker(t.getLocation()); return image; } } Sayyouhavetwopooledresources,suchasconnectionpoolsfortwodifferentdatabases.Resourcepoolsareusually implemented with semaphores (see Section 5.5.3) to facilitate blocking when the pool is empty. If a task requires connectionstobothdatabasesandthetworesourcesarenotalwaysrequestedinthesameorder,threadAcouldbe holdingaconnectiontodatabaseD1whilewaitingforaconnectiontodatabaseD2,andthreadBcouldbeholdinga connectiontoD2whilewaitingforaconnectiontoD1.(Thelargerthepoolsare,thelesslikelythisistooccur;ifeach poolhasNconnections,deadlockrequiresNsetsofcyclicallywaitingthreadsandalotofunluckytiming.) AnotherformofresourceͲbaseddeadlockisthreadͲstarvationdeadlock.WesawanexampleofthishazardinSection 8.1.1,whereataskthatsubmitsataskandwaitsforitsresultexecutesinasingleͲthreadedExecutor.Inthatcase,the firsttaskwillwaitforever,permanentlystallingthattaskandallotherswaitingtoexecuteinthatExecutor.Tasksthat wait for the results of other tasks are the primary source of threadͲstarvation deadlock; bounded pools and interdependenttasksdonotmixwell. 10.2.AvoidingandDiagnosingDeadlocks AprogramthatneveracquiresmorethanonelockatatimecannotexperiencelockͲorderingdeadlock.Ofcourse,thisis notalwayspractical,butifyoucangetawaywithit,it'salotlesswork.Ifyoumustacquiremultiplelocks,lockordering mustbeapartofyourdesign:trytominimizethenumberofpotentiallockinginteractions,andfollowanddocumenta lockͲorderingprotocolforlocksthatmaybeacquiredtogether.  134 JavaConcurrencyInPractice InprogramsthatusefineͲgrainedlocking,audityourcodefordeadlockfreedomusingatwoͲpartstrategy:first,identify wheremultiplelockscouldbeacquired(trytomakethisasmallset),andthenperformaglobalanalysisofallsuch instances to ensure that lock ordering is consistent across your entire program. Using open calls wherever possible simplifiesthisanalysissubstantially.WithnononͲopencalls,findinginstanceswheremultiplelocksareacquiredisfairly easy,eitherbycoderevieworbyautomatedbytecodeorsourcecodeanalysis. 10.2.1.TimedLockAttempts AnothertechniquefordetectingandrecoveringfromdeadlocksistousethetimedtryLockfeatureoftheexplicitLock classes(seeChapter13)insteadofintrinsiclocking.Whereintrinsiclockswaitforeveriftheycannotacquirethelock, explicitlocksletyouspecifyatimeoutafterwhichtryLockreturnsfailure.Byusingatimeoutthatismuchlongerthan youexpectacquiringthelocktotake,youcanregaincontrolwhensomethingunexpectedhappens.(Listing13.3on page280showsanalternativeimplementationoftransferMoneyusingthepolledtryLockwithretriesforprobabilistic deadlockavoidance.) When a timed lock attempt fails, you do not necessarily know why. Maybe there was a deadlock; maybe a thread erroneouslyenteredaninfiniteloopwhileholdingthatlock;ormaybesomeactivityisjustrunningalotslowerthanyou expected.Still,atleastyouhavetheopportunitytorecordthatyourattemptfailed,loganyusefulinformationabout whatyouweretryingtodo,andrestartthecomputationsomewhatmoregracefullythankillingtheentireprocess. Usingtimedlockacquisitiontoacquiremultiplelockscanbeeffectiveagainstdeadlockevenwhentimedlockingisnot usedconsistentlythroughouttheprogram.Ifalockacquisitiontimesout,youcanreleasethelocks,backoffandwait forawhile,andtryagain,possiblyclearingthedeadlockconditionandallowingtheprogramtorecover.(Thistechnique worksonlywhenthetwolocksareacquiredtogether;ifmultiplelocksareacquiredduetothenestingofmethodcalls, youcannotjustreleasetheouterlock,evenifyouknowyouholdit.) 10.2.2.DeadlockAnalysiswithThreadDumps Whilepreventingdeadlocksismostlyyourproblem,theJVMcanhelpidentifythemwhentheydohappenusingthread dumps.Athreaddumpincludesastacktraceforeachrunningthread,similartothestacktracethataccompaniesan exception.Threaddumpsalsoincludelockinginformation,suchaswhichlocksareheldbyeachthread,inwhichstack frametheywereacquired,andwhichlockablockedthreadiswaitingtoacquire.[4]Beforegeneratingathreaddump, theJVMsearchestheisͲwaitingͲforgraphforcyclestofinddeadlocks.Ifitfindsone,itincludesdeadlockinformation identifyingwhichlocksandthreadsareinvolved,andwhereintheprogramtheoffendinglockacquisitionsare. [4]Thisinformationisusefulfordebuggingevenwhenyoudon'thaveadeadlock;periodicallytriggeringthreaddumpsletsyouobserveyour program'slockingbehavior. Totriggerathreaddump,youcansendtheJVMprocessaSIGQUITsignal(kill -3)onUnixplatforms,orpressthe Ctrl-\keyonUnixorCtrl-BreakonWindowsplatforms.ManyIDEscanrequestathreaddumpaswell. If you are using the explicit Lock classes instead of intrinsic locking, Java 5.0 has no support for associating Lock informationwiththethreaddump;explicitLocksdonotshowupatallinthreaddumps.Java6doesincludethread dump support and deadlock detection with explicit Locks, but the information on where Locks are acquired is necessarilylessprecisethanforintrinsiclocks.Intrinsiclocksareassociatedwiththestackframeinwhichtheywere acquired;explicitLocksareassociatedonlywiththeacquiringthread. Listing10.7showsportionsofathreaddumptakenfromaproductionJ2EEapplication.Thefailurethatcausedthe deadlock involves three componentsa J2EE application, a J2EE container, and a JDBC driver, each from different vendors. (The names have been changed to protect the guilty.) All three were commercial products that had been throughextensivetestingcycles;eachhadabugthatwasharmlessuntiltheyallinteractedandcausedafatalserver failure. We'veshownonlytheportionofthethreaddumprelevanttoidentifyingthedeadlock.TheJVMhasdonealotofwork forusindiagnosingthedeadlockwhichlocksarecausingtheproblem,whichthreadsareinvolved,whichotherlocks they hold, and whether other threads are being indirectly inconvenienced. One thread holds the lock on the MumbleDBConnectionandiswaitingtoacquirethelockontheMumbleDBCallableStatement;theotherholdsthelock ontheMumbleDBCallableStatementandiswaitingforthelockontheMumbleDBConnection.  1356BPartIII:Liveness,Performance,andTesting Ͳ 22BChapter10.AvoidingLivenessHazards Listing10.7.PortionofThreadDumpAfterDeadlock. Found one Java-level deadlock: ============================= "ApplicationServerThread": waiting to lock monitor 0x080f0cdc (a MumbleDBConnection), which is held by "ApplicationServerThread" "ApplicationServerThread": waiting to lock monitor 0x080f0ed4 (a MumbleDBCallableStatement), which is held by "ApplicationServerThread" Java stack information for the threads listed above: "ApplicationServerThread": at MumbleDBConnection.remove_statement - waiting to lock <0x650f7f30> (a MumbleDBConnection) at MumbleDBStatement.close - locked <0x6024ffb0> (a MumbleDBCallableStatement) ... "ApplicationServerThread": at MumbleDBCallableStatement.sendBatch - waiting to lock <0x6024ffb0> (a MumbleDBCallableStatement) at MumbleDBConnection.commit - locked <0x650f7f30> (a MumbleDBConnection) ... TheJDBCdriverbeingusedhereclearlyhasalockͲorderingbug:differentcallchainsthroughtheJDBCdriveracquire multiple locks in different orders. But this problem would not have manifested itself were it not for another bug: multiplethreadsweretryingtousethesameJDBCConnectionatthesametime.Thiswasnothowtheapplicationwas supposedtoworkthedevelopersweresurprisedtoseethesameConnectionusedconcurrentlybytwothreads.There's nothingintheJDBCspecificationthatrequiresaConnectiontobethreadͲsafe,anditiscommontoconfineuseofa Connection to a single thread, as was intended here. This vendor tried to deliver a threadͲsafe JDBC driver, as evidencedbythesynchronizationonmultipleJDBCobjectswithinthedrivercode.Unfortunately,becausethevendor did not take lock ordering into account, the driver was prone to deadlock, but it was only the interaction of the deadlockͲpronedriverandtheincorrect Connectionsharingbytheapplicationthatdisclosedtheproblem.Because neitherbugwasfatalinisolation,bothpersisteddespiteextensivetesting. 10.3.OtherLivenessHazards While deadlock is the most widely encountered liveness hazard, there are several other liveness hazards you may encounter in concurrent programs including starvation, missed signals, and livelock. (Missed signals are covered in Section14.2.3.) 10.3.1.Starvation Starvationoccurswhenathreadisperpetuallydeniedaccesstoresourcesitneedsinordertomakeprogress;themost commonlystarvedresourceisCPUcycles.StarvationinJavaapplicationscanbecausedbyinappropriateuseofthread priorities.Itcanalsobecausedbyexecutingnonterminatingconstructs(infiniteloopsorresourcewaitsthatdonot terminate)withalockheld,sinceotherthreadsthatneedthatlockwillneverbeabletoacquireit. ThethreadprioritiesdefinedintheThreadAPIaremerelyschedulinghints.TheThreadAPIdefinestenprioritylevels thattheJVMcanmaptooperatingsystemschedulingprioritiesasitseesfit.ThismappingisplatformͲspecific,sotwo JavaprioritiescanmaptothesameOSpriorityononesystemanddifferentOSprioritiesonanother.Someoperating systemshavefewerthantenprioritylevels,inwhichcasemultipleJavaprioritiesmaptothesameOSpriority. Operatingsystemschedulersgotogreatlengthstoprovideschedulingfairnessandlivenessbeyondthatrequiredbythe Java Language Specification. In most Java applications, all application threads have the same priority, Thread. NORM_PRIORITY.Thethreadprioritymechanismisabluntinstrument,andit'snotalwaysobviouswhateffectchanging prioritieswillhave;boostingathread'sprioritymightdonothingormightalwayscauseonethreadtobescheduledin preferencetotheother,causingstarvation. Itisgenerallywisetoresistthetemptationtotweakthreadpriorities.Assoonasyoustartmodifyingpriorities,the behaviorofyourapplicationbecomesplatformͲspecificandyouintroducetheriskofstarvation.Youcanoftenspota program that is trying to recover from priority tweaking or other responsiveness problems by the presence of Thread.sleeporThread.yieldcallsinoddplaces,inanattempttogivemoretimetolowerͲprioritythreads.[5] [5]ThesemanticsofThread.yield(andThread.sleep(0))areundefined[JLS17.9];theJVMisfreetoimplementthemasnoͲopsortreat themasschedulinghints.Inparticular,theyarenotrequiredtohavethesemanticsofsleep(0)onUnixsystemsͲputthecurrentthreadatthe endoftherunqueueforthatpriority,yieldingtootherthreadsofthesamepriorityͲthoughsomeJVMsimplementyieldinthisway.  136 JavaConcurrencyInPractice Avoid the temptation to use thread priorities, since they increase platform dependence and can cause liveness problems.Mostconcurrentapplicationscanusethedefaultpriorityforallthreads. 10.3.2.PoorResponsiveness One step removed from starvation is poor responsiveness, which is not uncommon in GUI applications using backgroundthreads.Chapter9developedaframeworkforoffloadinglongͲrunningtasksontobackgroundthreadssoas not to freeze the user interface. CPUͲintensive background tasks can still affect responsiveness because they can competeforCPUcycleswiththeeventthread.Thisisonecasewherealteringthreadprioritiesmakessense;when computeͲintensivebackgroundcomputationswouldaffectresponsiveness.Iftheworkdonebyotherthreadsaretruly backgroundtasks,loweringtheirprioritycanmaketheforegroundtasksmoreresponsive. Poorresponsivenesscanalsobecausedbypoorlockmanagement.Ifathreadholdsalockforalongtime(perhaps whileiteratingalargecollectionandperformingsubstantialworkforeachelement),otherthreadsthatneedtoaccess thatcollectionmayhavetowaitaverylongtime. 10.3.3.Livelock Livelockisaformoflivenessfailureinwhichathread,whilenotblocked,stillcannotmakeprogressbecauseitkeeps retrying an operation that will always fail. Livelock often occurs in transactional messaging applications, where the messaginginfrastructurerollsbackatransactionifamessagecannotbeprocessedsuccessfully,andputsitbackatthe headofthequeue.Ifabuginthemessagehandlerforaparticulartypeofmessagecausesittofail,everytimethe messageisdequeuedandpassedtothebuggyhandler,thetransactionisrolledback.Sincethemessageisnowbackat theheadofthequeue,thehandleriscalledoverandoverwiththesameresult.(Thisissometimescalledthepoison messageproblem.)Themessagehandlingthreadisnotblocked,butitwillnevermakeprogresseither.Thisformof livelockoftencomesfromovereagererrorͲrecoverycodethatmistakenlytreatsanunrecoverableerrorasarecoverable one. Livelockcanalsooccurwhenmultiplecooperatingthreadschangetheirstateinresponsetotheothersinsuchaway thatnothreadcanevermakeprogress.Thisissimilartowhathappenswhentwooverlypolitepeoplearewalkingin oppositedirectionsinahallway:eachstepsoutoftheother'sway,andnowtheyareagainineachother'sway.Sothey bothstepasideagain,andagain,andagain... Thesolutionforthisvarietyoflivelockistointroducesomerandomnessintotheretrymechanism.Forexample,when twostationsinanEthernetnetworktrytosendapacketonthesharedcarrieratthesametime,thepacketscollide.The stationsdetectthecollision,andeachtriestosendtheirpacketagainlater.Iftheyeachretryexactlyonesecondlater, theycollideoverandover,andneitherpacketevergoesout,evenifthereisplentyofavailablebandwidth.Toavoid this,wemakeeachwaitanamountoftimethatincludesarandomcomponent.(TheEthernetprotocolalsoincludes exponentialbackͲoffafterrepeatedcollisions,reducingbothcongestionandtheriskofrepeatedfailurewithmultiple collidingstations.)RetryingwithrandomwaitsandbackͲoffscanbeequallyeffectiveforavoidinglivelockinconcurrent applications. Summary Livenessfailuresareaseriousproblembecausethereisnowaytorecoverfromthemshortofabortingtheapplication. ThemostcommonformoflivenessfailureislockͲorderingdeadlock.Avoidinglockorderingdeadlockstartsatdesign time:ensurethatwhenthreadsacquiremultiplelocks,theydosoinaconsistentorder.Thebestwaytodothisisby usingopencallsthroughoutyourprogram.Thisgreatlyreducesthenumberofplaceswheremultiplelocksareheldat once,andmakesitmoreobviouswherethoseplacesare.  1376BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability Chapter11.PerformanceandScalability Oneoftheprimaryreasonstousethreadsistoimproveperformance.[1]Usingthreadscanimproveresourceutilization by letting applications more easily exploit available processing capacity, and can improve responsiveness by letting applicationsbeginprocessingnewtasksimmediatelywhileexistingtasksarestillrunning. [1]Somemightarguethisistheonlyreasonweputupwiththecomplexitythreadsintroduce. Thischapterexplorestechniquesforanalyzing,monitoring,andimprovingtheperformanceofconcurrentprograms. Unfortunately, many of the techniques for improving performance also increase complexity, thus increasing the likelihood of safety and liveness failures. Worse, some techniques intended to improve performance are actually counterproductiveortradeonesortofperformanceproblemforanother.WhilebetterperformanceisoftendesirableͲ andimprovingperformancecanbeverysatisfyingͲsafetyalwayscomesfirst.Firstmakeyourprogramright,thenmake itfastͲandthenonlyifyourperformancerequirementsandmeasurementstellyouitneedstobefaster.Indesigninga concurrentapplication,squeezingoutthelastbitofperformanceisoftentheleastofyourconcerns. 11.1.ThinkingaboutPerformance Improvingperformancemeansdoingmoreworkwithfewerresources.Themeaningof"resources"canvary;foragiven activity,somespecificresourceisusuallyinshortestsupply,whetheritisCPUcycles,memory,networkbandwidth,I/O bandwidth,databaserequests,diskspace,oranynumberofotherresources.Whentheperformanceofanactivityis limitedbyavailabilityofaparticularresource,wesayitisboundbythatresource:CPUͲbound,databaseͲbound,etc. Whilethegoalmaybetoimproveperformanceoverall,usingmultiplethreadsalwaysintroducessomeperformance costscomparedtothesingleͲthreadedapproach.Theseincludetheoverheadassociatedwithcoordinatingbetween threads(locking,signaling,andmemorysynchronization),increasedcontextswitching,threadcreationandteardown, andschedulingoverhead.Whenthreadingisemployedeffectively,thesecostsaremorethanmadeupforbygreater throughput,responsiveness,orcapacity.Ontheotherhand,apoorlydesignedconcurrentapplicationcanperformeven worsethanacomparablesequentialone.[2] [2]Acolleagueprovidedthisamusinganecdote:hehadbeeninvolvedinthetestingofanexpensiveandcomplexapplicationthatmanagedits workviaatunablethreadpool.Afterthesystemwascomplete,testingshowedthattheoptimalnumberofthreadsforthepoolwas...1.This shouldhavebeenobviousfromtheoutset;thetargetsystemwasasingleͲCPUsystemandtheapplicationwasalmostentirelyCPUͲbound. Inusingconcurrencytoachievebetterperformance,wearetryingtodotwothings:utilizetheprocessingresourceswe have more effectively, and enable our program to exploit additional processing resources if they become available. From a performance monitoring perspective, this means we are looking to keep the CPUs as busy as possible. (Of course,thisdoesn'tmeanburningcycleswithuselesscomputation;wewanttokeeptheCPUsbusywithusefulwork.)If theprogramiscomputeͲbound,thenwemaybeabletoincreaseitscapacitybyaddingmoreprocessors;ifitcan'teven keeptheprocessorswehavebusy,addingmorewon'thelp.ThreadingoffersameanstokeeptheCPU(s)"hotter"by decomposingtheapplicationsothereisalwaysworktobedonebyanavailableprocessor. 11.1.1.PerformanceVersusScalability Applicationperformancecanbemeasuredinanumberofways,suchasservicetime,latency,throughput,efficiency, scalability,orcapacity.Someofthese(servicetime,latency)aremeasuresof"howfast"agivenunitofworkcanbe processedoracknowledged;others(capacity,throughput)aremeasuresof"howmuch"workcanbeperformedwitha givenquantityofcomputingresources. Scalability describes the ability to improve throughput or capacity when additional computing resources (such as additionalCPUs,memory,storage,orI/Obandwidth)areadded. Designing and tuning concurrent applications for scalability can be very different from traditional performance optimization.Whentuningforperformance,thegoalisusuallytodothesameworkwithlesseffort,suchasbyreusing previouslycomputedresultsthroughcachingorreplacinganO(n2)algorithmwithanO(nlogn)one.Whentuningfor scalability, you are instead trying to find ways to parallelize the problem so you can take advantage of additional processingresourcestodomoreworkwithmoreresources. ThesetwoaspectsofperformanceͲhowfastandhowmuchͲarecompletelyseparate,andsometimesevenatodds witheachother.Inordertoachievehigherscalabilityorbetterhardwareutilization,weoftenendupincreasingthe amountofworkdonetoprocesseachindividualtask,suchaswhenwedividetasksintomultiple"pipelined"subtasks.  138 JavaConcurrencyInPractice Ironically,manyofthetricksthatimproveperformanceinsingleͲthreadedprogramsarebadforscalability(seeSection 11.4.4foranexample). ThefamiliarthreeͲtierapplicationmodelͲinwhichpresentation,businesslogic,andpersistenceareseparatedandmay be handled by different systemsͲillustrates how improvements in scalability often come at the expense of performance. A monolithic application where presentation, business logic, and persistence are intertwined would almost certainly provide better performance for the first unit of work than would a wellͲfactored multitier implementationdistributedovermultiplesystems.Howcoulditnot?Themonolithicapplicationwouldnothavethe networklatencyinherentinhandingofftasksbetweentiers,norwouldithavetopaythecostsinherentinseparatinga computational process into distinct abstracted layers (such as queuing overhead, coordination overhead, and data copying). However, when the monolithic system reaches itsprocessing capacity, we could have aserious problem: it may be prohibitivelydifficulttosignificantlyincreasecapacity.Soweoftenaccepttheperformancecostsoflongerservicetime orgreatercomputingresourcesusedperunitofworksothatourapplicationcanscaletohandlegreaterloadbyadding moreresources. Ofthevariousaspectsofperformance,the"howmuch"aspectsͲscalability,throughput,andcapacityͲareusuallyof greaterconcernforserverapplicationsthanthe"howfast"aspects.(Forinteractiveapplications,latencytendstobe moreimportant,sothatusersneednotwaitforindicationsofprogressandwonderwhatisgoingon.)Thischapter focusesprimarilyonscalabilityratherthanrawsingleͲthreadedperformance. 11.1.2.EvaluatingPerformanceTradeoffs Nearlyallengineeringdecisionsinvolvesomeformoftradeoff.Usingthickersteelinabridgespanmayincreaseits capacityandsafety,butalsoitsconstructioncost.Whilesoftwareengineeringdecisionsdon'tusuallyinvolvetradeoffs betweenmoneyandrisktohumanlife,weoftenhavelessinformationwithwhichtomaketherighttradeoffs.For example, the "quicksort" algorithm is highly efficient for large data sets, but the less sophisticated "bubble sort" is actuallymoreefficientforsmalldatasets.Ifyouareaskedtoimplementanefficientsortroutine,youneedtoknow somethingaboutthesizesofdatasetsitwillhavetoprocess,alongwithmetricsthattellyouwhetheryouaretryingto optimizeaverageͲcasetime,worstͲcasetime,orpredictability.Unfortunately,thatinformationisoftennotpartofthe requirements given to the author of a library sort routine. This is one of the reasons why most optimizations are premature:theyareoftenundertakenbeforeaclearsetofrequirementsisavailable. Avoidprematureoptimization.Firstmakeitright,thenmakeitfastͲifitisnotalreadyfastenough. When making engineering decisions, sometimes you are trading one form of cost for another (service time versus memoryconsumption);sometimesyouaretradingcostforsafety.Safetydoesn'tnecessarilymeanrisktohumanlives, asitdidinthebridgeexample.ManyperformanceoptimizationscomeatthecostofreadabilityormaintainabilityͲthe more "clever" or nonͲobvious code is, the harder it is to understand and maintain. Sometimes optimizations entail compromisinggoodobjectͲorienteddesignprinciples,suchasbreakingencapsulation;sometimestheyinvolvegreater riskoferror,becausefasteralgorithmsareusuallymorecomplicated.(Ifyoucan'tspotthecostsorrisks,youprobably haven'tthoughtitthroughcarefullyenoughtoproceed.) Mostperformancedecisionsinvolvemultiplevariablesandarehighlysituational.Beforedecidingthatoneapproachis "faster"thananother,askyourselfsomequestions: x Whatdoyoumeanby"faster"? x Underwhatconditionswillthisapproachactuallybefaster?Underlightorheavyload?Withlargeorsmalldata sets?Canyousupportyouranswerwithmeasurements? x How often are these conditions likely to arise in your situation? Can you support your answer with measurements? x Isthiscodelikelytobeusedinothersituationswheretheconditionsmaybedifferent? x What hidden costs, such as increased development or maintenance risk, are you trading for this improved performance?Isthisagoodtradeoff? TheseconsiderationsapplytoanyperformanceͲrelatedengineeringdecision,butthisisabookaboutconcurrency.Why arewerecommendingsuchaconservativeapproachtooptimization?Thequestforperformanceisprobablythesingle greatestsourceofconcurrencybugs.Thebeliefthatsynchronizationwas"tooslow"ledtomanycleverͲlookingbut dangerous idioms for reducing synchronization (such as doubleͲchecked locking, discussed in Section 16.2.4), and is oftencitedasanexcusefornotfollowingtherulesregardingsynchronization.Becauseconcurrencybugsareamongthe .additionalcomputingresources countsincrease,evenasmallpercentageofserializedexecutionlimitshowmuchthroughputcanbeincreasedwith processors.(Utilizationisdefinedasthespeedupdividedbythenumberofprocessors.)Itisclearthatasprocessor Figure11.1showsthemaximumpossibleprocessorutilizationforvaryingdegreesofserialexecutionandnumbersof 9.2(at9%utilization).IttakesalotofinefficientlyutilizedCPUstonevergettothatfactoroften. canachieveatmostaspeedupof5.3(at53%utilization),andwith100processorsitcanachieveatmostaspeedupof Amdahl'slawalsoquantifiestheefficiencycostofserialization.Withtenprocessors,aprogramwith10%serialization available,andaprograminwhichtenpercentmustbeexecutedseriallycanbespedupbyatmostafactoroften. processingmustbeexecutedseriallycanbespeduponlybyafactoroftwo,regardlessofhowmanyprocessorsare AsNapproachesinfinity,themaximumspeedupconvergesto1/F,meaningthataprograminwhichfiftypercentofthe  ܰ ܨ൅ͳെܨ ܵ݌݁݁݀ݑ݌ ൑ ͳ serially,thenAmdahl'slawsaysthatonamachinewithNprocessors,wecanachieveaspeedupofatmost: the proportionofparallelizableandserialcomponents.IfFis thefractionofthe calculationthatmustbeexecuted Amdahl'slawdescribeshowmuchaprogramcantheoreticallybespedupbyadditionalcomputingresources,basedon Mostconcurrentprogramshavealotincommonwithfarming,consistingofamixofparallelizableandserialportions. potentialforparallelization. mustalsoensurethattheproblemisamenabletoparalleldecompositionandthatourprogrameffectivelyexploitsthis cropsgrowanyfaster.Ifoneofourprimaryreasonsforusingthreadsistoharnessthepowerofmultipleprocessors,we theharvestcanbecompleted.OthertasksarefundamentallyserialͲnonumberofadditionalworkerswillmakethe SomeproblemscanbesolvedfasterwithmoreresourcesͲthemoreworkersavailableforharvestingcrops,thefaster 11.2.Amdahl'sLaw been. CPUsbusy,thisisaverygoodwaytoevaluatewhetheryouneedperformancetuningorhoweffectiveyourtuninghas perfbarapplicationcangiveyouagoodpictureofhowbusytheCPUsare,andsinceyourgoalisusuallytokeepthe bottlenecks,butyoudon'thavetospendalotofmoneytofigureoutwhatyourprogramisdoing.Forexample,thefree There are sophisticated profiling tools on the market for measuring performance and tracking down performance Measure,don'tguess. evengetthedesiredbenefit. enoughͲyoudon'twanttopaythesecostsifyoudon'tneedtoͲandyoudefinitelydon'twanttopaythemifyoudon't achieved the desired improvements. The safety and maintenance risks associated with many optimizations are bad program in place using a realistic configuration and load profile. Measure again after tuning to verify that you've performance requirements (so you know both when to tune and when to stop tuning) and with a measurement is often incorrect. It is therefore imperative that any performance tuning exercise be accompanied by concrete intuitionofmanydevelopersaboutwhereaperformanceproblemliesorwhichapproachwillbefasterormorescalable Worse,whenyoutradesafetyforperformance,youmaygetneither. Especiallywhenit comestoconcurrency,the carefully. mostdifficulttotrackdownandeliminate,however,anythingthatrisksintroducingthemmustbeundertakenvery 1396BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability   140 JavaConcurrencyInPractice Figure11.1.MaximumUtilizationUnderAmdahl'sLawforVariousSerializationPercentages.   Chapter6exploredidentifyinglogicalboundariesfordecomposingapplicationsintotasks.Butinordertopredictwhat kindofspeedupispossiblefromrunningyourapplicationonamultiprocessorsystem,youalsoneedtoidentifythe sourcesofserializationinyourtasks. ImagineanapplicationwhereNthreadsexecutedoWorkinListing11.1,fetchingtasksfromasharedworkqueueand processingthem;assumethattasksdonotdependontheresultsorsideeffectsofothertasks.Ignoringforamoment howthetasksgetontothequeue,howwellwillthisapplicationscaleasweaddprocessors?Atfirstglance,itmay appear that the application is completely parallelizable: tasks do not wait for each other, and the more processors available,themoretaskscanbeprocessedconcurrently.However,thereisaserialcomponentaswellͲfetchingthetask from the work queue. The work queue is shared by all the worker threads, and it will require some amount of synchronizationtomaintainitsintegrityinthefaceofconcurrentaccess.Iflockingisusedtoguardthestateofthe queue,thenwhileonethreadisdequeingatask,otherthreadsthatneedtodequeuetheirnexttaskmustwaitͲandthis iswheretaskprocessingisserialized. The processing time of a single task includes not only the time to execute the task Runnable, but also the time to dequeuethetaskfromthesharedworkqueue.IftheworkqueueisaLinkedBlockingQueue,thedequeueoperation mayblocklessthanwithasynchronizedLinkedListbecauseLinkedBlockingQueueusesamorescalablealgorithm, butaccessinganyshareddatastructurefundamentallyintroducesanelementofserializationintoaprogram. Thisexamplealsoignoresanothercommonsourceofserialization:resulthandling.Allusefulcomputationsproduce somesortofresultorsideeffectifnot,theycanbeeliminatedasdeadcode.SinceRunnableprovidesfornoexplicit resulthandling,thesetasksmusthavesomesortofsideeffect,saywritingtheirresultstoalogfileorputtingthemina datastructure.Logfilesandresultcontainersareusuallysharedbymultipleworkerthreadsandthereforearealsoa sourceofserialization.Ifinsteadeachthreadmaintainsitsowndatastructureforresultsthataremergedafterallthe tasksareperformed,thenthefinalmergeisasourceofserialization.  1416BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability Listing11.1.SerializedAccesstoaTaskQueue. public class WorkerThread extends Thread { private final BlockingQueue queue; public WorkerThread(BlockingQueue queue) { this.queue = queue; } public void run() { while (true) { try { Runnable task = queue.take(); task.run(); } catch (InterruptedException e) { break; /* Allow thread to exit */ } } } }  Allconcurrentapplicationshavesomesourcesofserialization;ifyouthinkyoursdoesnot,lookagain. 11.2.1.Example:SerializationHiddeninFrameworks Toseehowserializationcanbehiddeninthestructureofanapplication,wecancomparethroughputasthreadsare added and infer differences in serialization based on observed differences in scalability. Figure 11.2 shows asimple applicationinwhichmultiplethreadsrepeatedlyremoveanelementfromashared Queueandprocessit,similarto Listing11.1.TheprocessingstepinvolvesonlythreadͲlocalcomputation.Ifathreadfindsthequeueisempty,itputsa batchofnewelementsonthequeuesothatotherthreadshavesomethingtoprocessontheirnextiteration.Accessing thesharedqueueclearlyentailssomedegreeofserialization,buttheprocessingstepisentirelyparallelizablesinceit involvesnoshareddata. Figure11.2.ComparingQueueImplementations. [Viewfullsizeimage]   ThecurvesinFigure11.2comparethroughputfortwothreadͲsafeQueueimplementations:aLinkedListwrappedwith synchronizedList,andaConcurrentLinkedQueue.Thetestswererunonan8ͲwaySparcV880systemrunningSolaris. Whileeachrunrepresentsthesameamountof"work",wecanseethatmerelychangingqueueimplementationscan haveabigimpactonscalability. The throughput of ConcurrentLinkedQueue continues to improve until it hits the number of processors and then remains mostly constant. On the other hand, the throughput of the synchronized LinkedList shows some improvementuptothreethreads,butthenfallsoffassynchronizationoverheadincreases.Bythetimeitgetstofouror fivethreads,contentionissoheavythateveryaccesstothequeuelockiscontendedandthroughputisdominatedby contextswitching. Thedifferenceinthroughputcomesfromdifferingdegreesofserializationbetweenthetwoqueueimplementations. ThesynchronizedLinkedListguardstheentirequeuestatewithasinglelockthatisheldforthedurationoftheoffer orremovecall;ConcurrentLinkedQueueusesasophisticatednonͲblockingqueuealgorithm(seeSection15.4.2)that  142 JavaConcurrencyInPractice usesatomicreferencestoupdateindividuallinkpointers.Inone,theentireinsertionorremovalisserialized;inthe other,onlyupdatestoindividualpointersareserialized. 11.2.2.ApplyingAmdahl'sLawQualitatively Amdahl's law quantifies the possible speedup when more computing resources are available, if we can accurately estimatethefractionofexecutionthatisserialized.Althoughmeasuringserializationdirectlycanbedifficult,Amdahl's lawcanstillbeusefulwithoutsuchmeasurement. Sinceourmentalmodelsareinfluencedbyourenvironment,manyofusareusedtothinkingthatamultiprocessor systemhastwoorfourprocessors,ormaybe(ifwe'vegotabigbudget)asmanyasafewdozen,becausethisisthe technologythathasbeenwidelyavailableinrecentyears.ButasmulticoreCPUsbecomemainstream,systemswillhave hundredsoreventhousandsofprocessors. [3]AlgorithmsthatseemscalableonafourͲwaysystemmayhavehidden scalabilitybottlenecksthathavejustnotyetbeenencountered. [3]Marketupdate:atthiswriting,SunisshippinglowͲendserversystemsbasedonthe8ͲcoreNiagaraprocessor,andAzulisshippinghighͲend serversystems(96,192,and384Ͳway)basedonthe24ͲcoreVegaprocessor. When evaluating an algorithm, thinking "in the limit" about what would happen with hundreds or thousands of processors can offer some insight into where scaling limits might appear. For example, Sections 11.4.2 and 11.4.3 discusstwotechniquesforreducinglockgranularity:locksplitting(splittingonelockintotwo)andlockstriping(splitting onelockintomany).LookingatthemthroughthelensofAmdahl'slaw,weseethatsplittingalockintwodoesnotget usveryfartowardsexploitingmanyprocessors,butlockstripingseemsmuchmorepromisingbecausethesizeofthe stripe set can be increased as processor count increases. (Of course, performance optimizations should always be consideredinlightofactualperformancerequirements;insomecases,splittingalockintwomaybeenoughtomeet therequirements.) 11.3.CostsIntroducedbyThreads SingleͲthreadedprogramsincurneitherschedulingnorsynchronizationoverhead,andneednotuselockstopreserve the consistency of data structures. Scheduling and interthread coordination have performance costs; for threads to offeraperformanceimprovement,theperformancebenefitsofparallelizationmustoutweighthecostsintroducedby concurrency. 11.3.1.ContextSwitching Ifthemainthreadistheonlyschedulablethread,itwillalmostneverbescheduledout.Ontheotherhand,ifthereare morerunnablethreadsthanCPUs,eventuallytheOSwillpreemptonethreadsothatanothercanusetheCPU.This causesacontextswitch,whichrequiressavingtheexecutioncontextofthecurrentlyrunningthreadandrestoringthe executioncontextofthenewlyscheduledthread. Contextswitchesarenotfree;threadschedulingrequiresmanipulatingshareddatastructuresintheOSandJVM.The OSandJVMusethesameCPUsyourprogramdoes;moreCPUtimespentinJVMandOScodemeanslessisavailablefor yourprogram.ButOSandJVMactivityisnottheonlycostofcontextswitches.Whenanewthreadisswitchedin,the dataitneedsisunlikelytobeinthelocalprocessorcache,soacontextswitchcausesaflurryofcachemisses,andthus threadsrunalittlemoreslowlywhentheyarefirstscheduled.Thisisoneofthereasonsthatschedulersgiveeach runnablethreadacertainminimumtimequantumevenwhenmanyotherthreadsarewaiting:itamortizesthecostof the contextswitchanditsconsequencesovermoreuninterruptedexecutiontime,improvingoverallthroughput (at somecosttoresponsiveness). Listing11.2.SynchronizationthathasNoEffect.Don'tDothis. synchronized (new Object()) { // do something } Whenathreadblocksbecauseitiswaitingforacontendedlock,theJVMusuallysuspendsthethreadandallowsittobe switchedout.Ifthreadsblockfrequently,theywillbeunabletousetheirfullschedulingquantum.Aprogramthatdoes more blocking (blocking I/O, waiting for contended locks, or waiting on condition variables) incurs more context switches than one that is CPUͲbound, increasing scheduling overhead and reducing throughput. (NonͲblocking algorithmscanalsohelpreducecontextswitches;seeChapter15.)  1436BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability Theactualcostofcontextswitchingvariesacrossplatforms,butagoodruleofthumbisthatacontextswitchcoststhe equivalentof5,000to10,000clockcycles,orseveralmicrosecondsonmostcurrentprocessors. The vmstat command on Unix systems and the perfmon tool on Windows systems report the number of context switchesandthepercentageoftimespentinthekernel.Highkernelusage(over10%)oftenindicatesheavyscheduling activity,whichmaybecausedbyblockingduetoI/Oorlockcontention. 11.3.2.MemorySynchronization The performance cost of synchronization comes from several sources. The visibility guarantees provided by synchronizedandvolatilemayentailusingspecialinstructionscalledmemorybarriersthatcanflushorinvalidate caches, flush hardware write buffers, and stall execution pipelines. Memory barriers may also have indirect performanceconsequencesbecausetheyinhibitothercompileroptimizations;mostoperationscannotbereordered withmemorybarriers. When assessing the performance impact of synchronization, it is important to distinguish between contended and uncontended synchronization. The synchronized mechanism is optimized for the uncontended case (volatile is alwaysuncontended),andatthiswriting,theperformancecostofa"fastͲpath"uncontendedsynchronizationranges from 20 to 250 clock cycles for most systems. While this is certainly not zero, the effect of needed, uncontended synchronizationisrarelysignificantinoverallapplicationperformance,andthealternativeinvolvescompromisingsafety andpotentiallysigningyourself(oryoursuccessor)upforsomeverypainfulbughuntinglater. ModernJVMscanreducethecostofincidentalsynchronizationbyoptimizingawaylockingthatcanbeprovenneverto contend. If a lock object is accessible only to the current thread, the JVM is permitted to optimize away a lock acquisition because there is no way another thread could synchronize on the same lock. For example, the lock acquisitioninListing11.2canalwaysbeeliminatedbytheJVM. MoresophisticatedJVMscanuseescapeanalysistoidentifywhenalocalobjectreferenceisneverpublishedtothe heapandisthereforethreadͲlocal.IngetStoogeNamesinListing11.3,theonlyreferencetotheLististhelocalvariable stooges, and stackͲconfined variables are automatically threadͲlocal. A naive execution of getStoogeNames would acquireandreleasethelockontheVectorfourtimes,onceforeachcalltoaddortoString.However,asmartruntime compilercaninlinethesecallsandthenseethatstoogesanditsinternalstateneverescape,andthereforethatallfour lockacquisitionscanbeeliminated.[4] [4]Thiscompileroptimization,calledlockelision,isperformedbytheIBMJVMandisexpectedinHotSpotasofJava7. Listing11.3.CandidateforLockElision. public String getStoogeNames() { List stooges = new Vector(); stooges.add("Moe"); stooges.add("Larry"); stooges.add("Curly"); return stooges.toString(); } Evenwithoutescapeanalysis,compilerscanalsoperformlockcoarsening,themergingofadjacentsynchronizedblocks usingthesamelock.ForgetStooge-Names,aJVMthatperformslockcoarseningmightcombinethethreecallstoadd and the call to toString into a single lock acquisition and release, using heuristics on the relative cost of synchronizationversustheinstructionsinsidethesynchronizedblock.[5]Notonlydoesthisreducethesynchronization overhead,butitalsogivestheoptimizeramuchlargerblocktoworkwith,likelyenablingotheroptimizations. [5] A smart dynamic compiler can figure out that this method always returns the same string, and after the first execution recompile getStoogeNamestosimplyreturnthevaluereturnedbythefirstexecution. Don'tworryexcessivelyaboutthecostofuncontendedsynchronization.Thebasicmechanismisalreadyquitefast,and JVMscanperformadditionaloptimizationsthatfurtherreduceoreliminatethecost.Instead,focusoptimizationefforts onareaswherelockcontentionactuallyoccurs. Synchronizationbyonethreadcanalsoaffecttheperformanceofotherthreads.Synchronizationcreatestrafficonthe sharedmemorybus;thisbushasalimitedbandwidthandissharedacrossallprocessors.Ifthreadsmustcompetefor synchronizationbandwidth,allthreadsusingsynchronizationwillsuffer.[6] [6] This aspect is sometimes used to argue against the use of nonͲblocking algorithms without some sort of backoff, because under heavy contention,nonͲblockingalgorithmsgeneratemoresynchronizationtrafficthanlockͲbasedones.SeeChapter15.  144 JavaConcurrencyInPractice 11.3.3.Blocking UncontendedsynchronizationcanbehandledentirelywithintheJVM(Baconetal.,1998);contendedsynchronization mayrequireOSactivity,whichaddstothecost.Whenlockingiscontended,thelosingthread(s)mustblock.TheJVM canimplementblockingeitherviaspinͲwaiting(repeatedlytryingtoacquirethelockuntilitsucceeds)orbysuspending theblockedthreadthroughtheoperatingsystem.Whichismoreefficientdependsontherelationshipbetweencontext switchoverheadandthetimeuntilthelockbecomesavailable;spinͲwaitingispreferableforshortwaitsandsuspension ispreferableforlongwaits.SomeJVMschoosebetweenthetwoadaptivelybasedonprofilingdataofpastwaittimes, butmostjustsuspendthreadswaitingforalock. Suspendingathreadbecauseitcouldnotgetalock,orbecauseitblockedonaconditionwaitorblockingI/Ooperation, entailstwoadditionalcontextswitchesandalltheattendantOSandcacheactivity:theblockedthreadisswitchedout beforeitsquantumhasexpired,andisthenswitchedbackinlaterafterthelockorotherresourcebecomesavailable. (Blockingduetolockcontentionalsohasacostforthethreadholdingthelock:whenitreleasesthelock,itmustthen asktheOStoresumetheblockedthread.) 11.4.ReducingLockContention We'veseenthatserializationhurtsscalabilityandthatcontextswitcheshurtperformance.Contendedlockingcauses both,soreducinglockcontentioncanimprovebothperformanceandscalability. Accesstoresourcesguardedbyanexclusivelockisserializedonlyonethreadatatimemayaccessit.Ofcourse,weuse locksforgoodreasons,suchaspreventingdatacorruption,butthissafetycomesataprice.Persistentcontentionfora locklimitsscalability. Theprincipalthreattoscalabilityinconcurrentapplicationsistheexclusiveresourcelock. Twofactorsinfluencethelikelihoodofcontentionforalock:howoftenthatlockisrequestedandhowlongitisheld onceacquired.[7]Iftheproductofthesefactorsissufficientlysmall,thenmostattemptstoacquirethelockwill be uncontended, and lock contention will not pose a significant scalability impediment. If, however, the lock is in sufficientlyhighdemand,threadswillblockwaitingforit;intheextremecase,processorswillsitidleeventhoughthere isplentyofworktodo. [7]ThisisacorollaryofLittle'slaw,aresultfromqueuingtheorythatsays"theaveragenumberofcustomersinastablesystemisequaltotheir averagearrivalratemultipliedbytheiraveragetimeinthesystem".(Little,1961) Therearethreewaystoreducelockcontention: x Reducethedurationforwhichlocksareheld; x Reducethefrequencywithwhichlocksarerequested;or x Replaceexclusivelockswithcoordinationmechanismsthatpermitgreaterconcurrency. 11.4.1.NarrowingLockScope("Getin,GetOut") Aneffectivewaytoreducethelikelihoodofcontentionistoholdlocksasbrieflyaspossible.Thiscanbedonebymoving code that doesn't require the lock out of synchronized blocks, especially for expensive operations and potentially blockingoperationssuchasI/O. It is easy to see how holding a "hot" lock for too long can limit scalability; we saw an example of this in SynchronizedFactorizerinChapter2.Ifanoperationholdsalockfor2millisecondsandeveryoperationrequiresthat lock,throughputcanbenogreaterthan500operationspersecond,nomatterhowmanyprocessorsareavailable. Reducingthetimethelockisheldto1millisecondimprovesthelockͲinducedthroughputlimitto1000operationsper second.[8] [8]Actually,thiscalculationunderstatesthecostofholdinglocksfortoolongbecauseitdoesn'ttakeintoaccountthecontextswitchoverhead generatedbyincreasedlockcontention. AttributeStoreinListing11.4showsanexampleofholdingalocklongerthannecessary.TheuserLocationMatches methodlooksuptheuser'slocationinaMapandusesregularexpressionmatchingtoseeiftheresultingvaluematches thesuppliedpattern.TheentireuserLocationMatchesmethodissynchronized,buttheonlyportionofthecodethat actuallyneedsthelockisthecalltoMap.get.  1456BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability Listing11.4.HoldingaLockLongerthanNecessary. @ThreadSafe public class AttributeStore { @GuardedBy("this") private final Map attributes = new HashMap(); public synchronized boolean userLocationMatches(String name, String regexp) { String key = "users." + name + ".location"; String location = attributes.get(key); if (location == null) return false; else return Pattern.matches(regexp, location); } } BetterAttributeStoreinListing11.5rewritesAttributeStoretoreducesignificantlythelockduration.Thefirststep istoconstructtheMapkeyassociatedwiththeuser'slocation,astringoftheformusers.name.location.Thisentails instantiatingaStringBuilderobject,appendingseveralstringstoit,andinstantiatingtheresultasaString.Afterthe location has been retrieved, the regular expression is matched against the resulting location string. Because constructingthekeystringandprocessingtheregularexpressiondonotaccesssharedstate,theyneednotbeexecuted withthelockheld.BetterAttributeStorefactorsthesestepsoutofthesynchronizedblock,thusreducingthetime thelockisheld. Listing11.5.ReducingLockDuration. @ThreadSafe public class BetterAttributeStore { @GuardedBy("this") private final Map attributes = new HashMap(); public boolean userLocationMatches(String name, String regexp) { String key = "users." + name + ".location"; String location; synchronized (this) { location = attributes.get(key); } if (location == null) return false; else return Pattern.matches(regexp, location); } } Reducing the scope of the lock in userLocationMatches substantially reduces the number of instructions that are executed with the lock held. By Amdahl's law, this removes an impediment to scalability because the amount of serializedcodeisreduced. Because AttributeStore has only one state variable, attributes, we can improve it further by the technique of delegatingthreadsafety(Section4.3).ByreplacingattributeswithathreadͲsafeMap(aHashtable,synchronizedMap, or ConcurrentHashMap), AttributeStorecandelegateallitsthreadsafetyobligationstotheunderlyingthreadͲsafe collection. This eliminates the need for explicit synchronization in AttributeStore, reduces the lock scope to the durationoftheMapaccess,andremovestheriskthatafuturemaintainerwillunderminethreadsafetybyforgettingto acquiretheappropriatelockbeforeaccessingattributes. Whileshrinkingsynchronizedblockscanimprovescalability,asynchronizedblockcanbetoosmalloperationsthat need to be atomic (such updating multiple variables that participate in an invariant) must be contained in a single synchronized block. And because the cost of synchronization is nonzero, breaking one synchronized block into multiple synchronized blocks (correctness permitting) at some point becomes counterproductive in terms of performance.[9]TheidealbalanceisofcourseplatformͲdependent,butinpracticeitmakessensetoworryaboutthe sizeofasynchronizedblockonlywhenyoucanmove"substantial"computationorblockingoperationsoutofit. [9]IftheJVMperformslockcoarsening,itmayundothesplittingofsynchronizedblocksanyway. 11.4.2.ReducingLockGranularity Theotherwaytoreducethefractionoftimethatalockisheld(andthereforethelikelihoodthatitwillbecontended)is tohavethreadsaskforitlessoften.Thiscanbeaccomplishedbylocksplittingandlockstriping,whichinvolveusing  146 JavaConcurrencyInPractice separate locks to guard multiple independent state variables previously guarded by a single lock. These techniques reducethegranularityatwhichlockingoccurs,potentiallyallowinggreaterscalabilitybutusingmorelocksalsoincreases theriskofdeadlock. Asathoughtexperiment,imaginewhatwouldhappeniftherewasonlyonelockfortheentireapplicationinsteadofa separatelockforeachobject.Thenexecutionofallsynchronizedblocks,regardlessoftheirlock,wouldbeserialized. Withmanythreadscompetingforthegloballock,thechancethattwothreadswantthelockatthesametimeincreases, resultinginmorecontention.Soiflockrequestswereinsteaddistributedoveralargersetoflocks,therewouldbeless contention.Fewerthreadswouldbeblockedwaitingforlocks,thusincreasingscalability. Ifalockguardsmorethanoneindependentstatevariable,youmaybeabletoimprovescalabilitybysplittingitinto multiplelocksthateachguarddifferentvariables.Thisresultsineachlockbeingrequestedlessoften. ServerStatusinListing11.6showsaportionofthemonitoringinterfaceforadatabaseserverthatmaintainsthesetof currentlyloggedͲonusersandthesetofcurrentlyexecutingqueries.Asauserlogsonorofforqueryexecutionbegins or ends, the ServerStatus object is updated by calling the appropriate add or remove method. The two types of informationarecompletelyindependent;ServerStatuscouldevenbesplitintotwoseparateclasseswithnolossof functionality. InsteadofguardingbothusersandquerieswiththeServerStatuslock,wecaninsteadguardeachwithaseparate lock,asshowninListing11.7.Aftersplittingthelock,eachnewfinerͲgrainedlockwillseelesslockingtrafficthanthe originalcoarserlockwouldhave.(DelegatingtoathreadͲsafeSetimplementationforusersandqueriesinsteadof usingexplicitsynchronizationwouldimplicitlyprovidelocksplitting,aseachSetwoulduseadifferentlocktoguardits state.) Splittingalockintotwooffersthegreatestpossibilityforimprovementwhenthelockisexperiencingmoderatebutnot heavycontention.Splittinglocksthatareexperiencinglittlecontentionyieldslittlenetimprovementinperformanceor throughput,althoughitmightincreasetheloadthresholdatwhichperformancestartstodegradeduetocontention. Splittinglocksexperiencingmoderatecontentionmightactuallyturnthemintomostlyuncontendedlocks,whichisthe mostdesirableoutcomeforbothperformanceandscalability. Listing11.6.CandidateforLockSplitting. @ThreadSafe public class ServerStatus { @GuardedBy("this") public final Set users; @GuardedBy("this") public final Set queries; ... public synchronized void addUser(String u) { users.add(u); } public synchronized void addQuery(String q) { queries.add(q); } public synchronized void removeUser(String u) { users.remove(u); } public synchronized void removeQuery(String q) { queries.remove(q); } } Listing11.7.ServerStatusRefactoredtoUseSplitLocks. @ThreadSafe public class ServerStatus { @GuardedBy("users") public final Set users; @GuardedBy("queries") public final Set queries; ... public void addUser(String u) { synchronized (users) { users.add(u); } } public void addQuery(String q) { synchronized (queries) { queries.add(q); } } // remove methods similarly refactored to use split locks } 11.4.3.LockStriping Splittingaheavilycontendedlockintotwoislikelytoresultintwoheavilycontendedlocks.Whilethiswillproducea small scalability improvement by enabling two threads to execute concurrently instead of one, it still does not  1476BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability dramaticallyimproveprospectsforconcurrencyonasystemwithmanyprocessors.Thelocksplittingexampleinthe ServerStatusclassesdoesnotofferanyobviousopportunityforsplittingthelocksfurther. Locksplittingcansometimesbeextendedtopartitionlockingonavariablesizedsetofindependentobjects,inwhich caseitiscalledlockstriping.Forexample,theimplementationofConcurrentHashMapusesanarrayof16locks,eachof whichguards1/16ofthehashbuckets;bucketNisguardedbylockNmod16.Assumingthehashfunctionprovides reasonablespreadingcharacteristicsandkeysareaccesseduniformly,thisshouldreducethedemandforanygivenlock byapproximatelyafactorof16.ItisthistechniquethatenablesConcurrentHashMaptosupportupto16concurrent writers. (The number of locks could be increased to provide even better concurrency under heavy access on highͲ processorͲcountsystems,butthenumberofstripesshouldbeincreasedbeyondthedefaultof16onlywhenyouhave evidencethatconcurrentwritersaregeneratingenoughcontentiontowarrantraisingthelimit.) Oneofthedownsidesoflockstripingisthatlockingthecollectionforexclusiveaccessismoredifficultandcostlythan withasinglelock.Usuallyanoperationcanbeperformedbyacquiringatmostonelock,butoccasionallyyouneedto locktheentirecollection,aswhenConcurrentHashMapneedstoexpandthemapandrehashthevaluesintoalargerset ofbuckets.Thisistypicallydonebyacquiringallofthelocksinthestripeset.[10] [10]Theonlywaytoacquireanarbitrarysetofintrinsiclocksisviarecursion. StripedMapinListing11.8illustratesimplementingahashͲbasedmapusinglockstriping.ThereareN_LOCKSlocks,each guardingasubsetofthebuckets.Mostmethods,likeget,needacquireonlyasinglebucketlock.Somemethodsmay need to acquire all the locks but, as in the implementation for clear, may not need to acquire them all simultaneously.[11] [11]ClearingtheMapinthiswayisnotatomic,sothereisnotnecessarilyatimewhentheStriped-Mapisactuallyemptyifotherthreadsare concurrentlyaddingelements;makingtheoperationatomicwouldrequireacquiringallthelocksatonce.However,forconcurrentcollectionsthat clientstypicallycannotlockforexclusiveaccess,theresultofmethodslikesizeorisEmptymaybeoutofdatebythetimetheyreturnanyway, sothisbehavior,whileperhapssomewhatsurprising,isusuallyacceptable. 11.4.4.AvoidingHotFields Locksplittingandlockstripingcanimprovescalabilitybecausetheyenabledifferentthreadstooperateondifferent data (or different portions of the same data structure) without interfering with each other. A program that would benefitfromlocksplittingnecessarilyexhibitscontentionforalockmoreoftenthanforthedataguardedbythatlock.If alockguardstwoindependentvariablesXandY,andthreadAwantstoaccessXwhileBwantstoaccessY(aswouldbe thecaseifonethreadcalledaddUserwhileanothercalledaddQueryinServerStatus),thenthetwothreadsarenot contendingforanydata,eventhoughtheyarecontendingforalock.  148 JavaConcurrencyInPractice Listing11.8.HashǦbasedMapUsingLockStriping. @ThreadSafe public class StripedMap { // Synchronization policy: buckets[n] guarded by locks[n%N_LOCKS] private static final int N_LOCKS = 16; private final Node[] buckets; private final Object[] locks; private static class Node { ... } public StripedMap(int numBuckets) { buckets = new Node[numBuckets]; locks = new Object[N_LOCKS]; for (int i = 0; i < N_LOCKS; i++) locks[i] = new Object(); } private final int hash(Object key) { return Math.abs(key.hashCode() % buckets.length); } public Object get(Object key) { int hash = hash(key); synchronized (locks[hash % N_LOCKS]) { for (Node m = buckets[hash]; m != null; m = m.next) if (m.key.equals(key)) return m.value; } return null; } public void clear() { for (int i = 0; i < buckets.length; i++) { synchronized (locks[i % N_LOCKS]) { buckets[i] = null; } } } ... } Lockgranularitycannotbereducedwhentherearevariablesthatarerequiredforeveryoperation.Thisisyetanother areawhererawperformanceandscalabilityareoftenatoddswitheachother;commonoptimizationssuchascaching frequentlycomputedvaluescanintroduce"hotfields"thatlimitscalability. IfyouwereimplementingHashMap,youwouldhaveachoiceofhowsizecomputesthenumberofentriesintheMap. Thesimplestapproachistocountthenumberofentrieseverytimeitiscalled.Acommonoptimizationistoupdatea separatecounterasentriesareaddedorremoved;thisslightlyincreasesthecostofaputorremoveoperationtokeep thecounterupͲtoͲdate,butreducesthecostofthesizeoperationfromO(n)toO(1). Keeping a separate count to speed up operations like size and isEmpty works fine for a singleͲthreaded or fully synchronized implementation, but makes it much harder to improve the scalability of the implementation because everyoperationthatmodifiesthemapmustnowupdatethesharedcounter.Evenifyouuselockstripingforthehash chains,synchronizingaccesstothecounterreintroducesthescalabilityproblemsofexclusivelocking.Whatlookedlikea performanceoptimizationͲcachingtheresultsofthesizeoperationͲhasturnedintoascalabilityliability.Inthiscase, thecounteriscalledahotfieldbecauseeverymutativeoperationneedstoaccessit. ConcurrentHashMapavoidsthisproblembyhavingsizeenumeratethestripesandaddupthenumberofelementsin eachstripe,insteadofmaintainingaglobalcount.Toavoidenumeratingeveryelement,ConcurrentHashMapmaintains aseparatecountfieldforeachstripe,alsoguardedbythestripelock.[12] [12]Ifsizeiscalledfrequentlycomparedtomutativeoperations,stripeddatastructurescanoptimizeforthisbycachingthecollectionsizeina volatile whenever size is called and invalidating the cache (setting it toͲ1) whenever the collection is modified. If the cached value is nonnegativeonentrytosize,itisaccurateandcanbereturned;otherwiseitisrecomputed. 11.4.5.AlternativestoExclusiveLocks Athirdtechniqueformitigatingtheeffectoflockcontentionistoforegotheuseofexclusivelocksinfavorofamore concurrencyͲfriendlymeansofmanagingsharedstate.Theseincludeusingtheconcurrentcollections,readͲwritelocks, immutableobjectsandatomicvariables. ReadWriteLock(seeChapter13)enforcesamultipleͲreader,singleͲwriterlockingdiscipline:morethanonereadercan accessthesharedresourceconcurrentlysolongasnoneofthemwantstomodifyit,butwritersmustacquirethelock  1496BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability exclusively.ForreadͲmostlydatastructures,ReadWriteLockcanoffergreaterconcurrencythanexclusivelocking;for readͲonlydatastructures,immutabilitycaneliminatetheneedforlockingentirely. Atomicvariables(seeChapter15)offerameansofreducingthecostofupdating"hotfields"suchasstatisticscounters, sequencegenerators,orthereferencetothefirstnodeinalinkeddatastructure.(WeusedAtomicLongtomaintainthe hitcounterintheservletexamplesinChapter2.)TheatomicvariableclassesprovideveryfineͲgrained(andtherefore morescalable)atomicoperationsonintegersorobjectreferences,andareimplementedusinglowͲlevelconcurrency primitives(suchascompareͲandͲswap)providedbymostmodernprocessors.Ifyourclasshasasmallnumberofhot fields that do not participate in invariants with other variables, replacing them with atomic variables may improve scalability.(ChangingyouralgorithmtohavefewerhotfieldsmightimprovescalabilityevenmoreͲatomicvariables reducethecostofupdatinghotfields,buttheydon'teliminateit.) 11.4.6.MonitoringCPUUtilization Whentestingforscalability,thegoalisusuallytokeeptheprocessorsfullyutilized.Toolslikevmstatandmpstaton UnixsystemsorperfmononWindowssystemscantellyoujusthow"hot"theprocessorsarerunning. IftheCPUsareasymmetricallyutilized(someCPUsarerunninghotbutothersarenot)yourfirstgoalshouldbetofind increasedparallelisminyourprogram.Asymmetricutilizationindicatesthatmostofthecomputationisgoingonina smallsetofthreads,andyourapplicationwillnotbeabletotakeadvantageofadditionalprocessors. IftheCPUsarenotfullyutilized,youneedtofigureoutwhy.Thereareseverallikelycauses: Insufficientload.Itmaybethattheapplicationbeingtestedisjustnotsubjectedtoenoughload.Youcantestforthisby increasingtheloadandmeasuringchangesinutilization,responsetime,orservicetime.Generatingenoughloadto saturateanapplicationcanrequiresubstantialcomputerpower;theproblemmaybethattheclientsystems,notthe systembeingtested,arerunningatcapacity. I/OͲbound. You can determine whether an application is diskͲbound using iostat or perfmon, and whether it is bandwidthͲlimitedbymonitoringtrafficlevelsonyournetwork. Externallybound.Ifyourapplicationdependsonexternalservicessuchasadatabaseorwebservice,thebottleneck maynotbeinyourcode.Youcantestforthisbyusingaprofilerordatabaseadministrationtoolstodeterminehow muchtimeisbeingspentwaitingforanswersfromtheexternalservice. Lockcontention.Profilingtoolscantellyouhowmuchlockcontentionyourapplicationisexperiencingandwhichlocks are"hot".Youcanoftengetthesameinformationwithoutaprofilerthroughrandomsampling,triggeringafewthread dumpsandlookingforthreadscontendingforlocks.Ifathreadisblockedwaitingforalock,theappropriatestackframe inthethreaddumpindicates"waitingtolockmonitor..."Locksthataremostlyuncontendedrarelyshowupina threaddump;aheavilycontendedlockwillalmostalwayshaveatleastonethreadwaitingtoacquireitandsowill frequentlyappearinthreaddumps. IfyourapplicationiskeepingtheCPUssufficientlyhot,youcanusemonitoringtoolstoinferwhetheritwouldbenefit fromadditionalCPUs.Aprogramwithonlyfourthreadsmaybeabletokeepa4Ͳwaysystemfullyutilized,butisunlikely toseeaperformanceboostifmovedtoan8Ͳwaysystemsincetherewouldneedtobewaitingrunnablethreadstotake advantageoftheadditionalprocessors.(Youmayalsobeabletoreconfiguretheprogramtodivideitsworkloadover morethreads,suchasadjustingathreadpoolsize.)Oneofthecolumnsreportedbyvmstatisthenumberofthreads thatarerunnablebutnotcurrentlyrunningbecauseaCPUisnotavailable;ifCPUutilizationishighandtherearealways runnablethreadswaitingforaCPU,yourapplicationwouldprobablybenefitfrommoreprocessors. 11.4.7.JustSayNotoObjectPooling In early JVM versions, object allocation and garbage collection were slow,[13] but their performance has improved substantiallysincethen.Infact,allocationinJavaisnowfasterthan mallocisinC:thecommoncodepathfor new ObjectinHotSpot1.4.xand5.0isapproximatelytenmachineinstructions. [13]AswaseverythingelseͲsynchronization,graphics,JVMstartup,reflectionͲpredictablysointhefirstversionofanexperimentaltechnology. Toworkaround"slow"objectlifecycles,manydevelopersturnedtoobjectpooling,whereobjectsarerecycledinstead ofbeinggarbagecollectedandallocatedanewwhenneeded.Eventakingintoaccountitsreducedgarbagecollection overhead,objectpoolinghasbeenshowntobeaperformanceloss[14]forallbutthemostexpensiveobjects(anda seriouslossforlightͲandmediumͲweightobjects)insingleͲthreadedprograms(Click,2005).  150 JavaConcurrencyInPractice [14]InadditiontobeingalossintermsofCPUcycles,objectpoolinghasanumberofotherproblems,amongthemthechallengeofsettingpool sizescorrectly(toosmall,andpoolinghasnoeffect;toolarge,anditputspressureonthegarbagecollector,retainingmemorythatcouldbeused moreeffectivelyforsomethingelse);theriskthatanobjectwillnotbeproperlyresettoitsnewlyallocatedstate,introducingsubtlebugs;therisk that a thread will return an object to the pool but continue using it; and that it makes more work for generational garbage collectors by encouragingapatternofoldͲtoͲyoungreferences. In concurrent applications, pooling fares even worse. When threads allocate new objects, very little interͲthread coordinationisrequired,asallocatorstypicallyusethreadͲlocalallocationblockstoeliminatemostsynchronizationon heapdatastructures.Butifthosethreadsinsteadrequestanobjectfromapool,somesynchronizationisnecessaryto coordinateaccesstothepooldatastructure,creatingthepossibilitythatathreadwillblock.Becauseblockingathread duetolockcontentionishundredsoftimesmoreexpensivethananallocation,evenasmallamountofpoolͲinduced contentionwouldbeascalabilitybottleneck. (Evenanuncontendedsynchronizationisusuallymoreexpensivethan allocating an object.) This is yet another technique intended as a performance optimization but that turned into a scalabilityhazard.Poolinghasitsuses,[15]butisoflimitedutilityasaperformanceoptimization. [15]Inconstrainedenvironments,suchassomeJ2MEorRTSJtargets,objectpoolingmaystillberequiredforeffectivememorymanagementorto manageresponsiveness. Allocatingobjectsisusuallycheaperthansynchronizing. 11.5.Example:ComparingMapPerformance ThesingleͲthreadedperformanceofConcurrentHashMapisslightlybetterthanthatofasynchronizedHashMap,butitis in concurrent use that it really shines. The implementation of ConcurrentHashMap assumes the most common operation is retrieving a value that already exists, and is therefore optimized to provide highest performance and concurrencyforsuccessfulgetoperations. ThemajorscalabilityimpedimentforthesynchronizedMapimplementationsisthatthereisasinglelockfortheentire map,soonlyonethreadcanaccessthemapatatime.Ontheotherhand,ConcurrentHashMapdoesnolockingformost successfulreadoperations,anduseslockstripingforwriteoperationsandthosefewreadoperationsthatdorequire locking.Asaresult,multiplethreadscanaccesstheMapconcurrentlywithoutblocking. Figure 11.3 illustrates the differences in scalability between several Map implementations: ConcurrentHashMap, ConcurrentSkipListMap,andHashMapandtreeMapwrappedwithsynchronizedMap.ThefirsttwoarethreadͲsafeby design;thelattertwoaremadethreadͲsafebythesynchronizedwrapper.Ineachrun,Nthreadsconcurrentlyexecutea tightloopthatselectsarandomkeyandattemptstoretrievethevaluecorrespondingtothatkey.Ifthevalueisnot present,itisaddedtotheMapwithprobabilityp=.6,andifitispresent,isremovedwithprobabilityp=.02.Thetests wererununderapreͲreleasebuildofJava6onan8ͲwaySparcV880,andthegraphdisplaysthroughputnormalizedto theonethreadcaseforConcurrentHashMap.(Thescalabilitygapbetweentheconcurrentandsynchronizedcollectionsis evenlargeronJava5.0.) ThedataforConcurrentHashMapandConcurrentSkipListMapshowsthattheyscalewelltolargenumbersofthreads; throughputcontinuestoimproveasthreadsareadded.WhilethenumbersofthreadsinFigure11.3maynotseem large,thistestprogramgeneratesmorecontentionperthreadthanatypicalapplicationbecauseitdoeslittleotherthan poundontheMap;arealprogramwoulddoadditionalthreadͲlocalworkineachiteration. Figure11.3.ComparingScalabilityofMapImplementations. [Viewfullsizeimage]   1516BPartIII:Liveness,Performance,andTesting Ͳ 23BChapter11.PerformanceandScalability  The numbers for the synchronized collections are not as encouraging. Performance for the oneͲthread case is comparable to ConcurrentHashMap, but once the load transitions from mostly uncontended to mostly contendedͲ whichhappenshereattwothreadsͲthesynchronizedcollectionssufferbadly.Thisiscommonbehaviorforcodewhose scalabilityislimitedbylockcontention.Solongascontentionislow,timeperoperationisdominatedbythetimeto actuallydotheworkandthroughputmayimproveasthreadsareadded.Oncecontentionbecomessignificant,timeper operation is dominated by context switch and scheduling delays, and adding more threads has little effect on throughput. 11.6.ReducingContextSwitchOverhead Manytasksinvolveoperationsthatmayblock;transitioningbetweentherunningandblockedstatesentailsacontext switch.Onesourceofblockinginserverapplicationsisgeneratinglogmessagesinthecourseofprocessingrequests;to illustratehowthroughputcanbeimprovedbyreducingcontextswitches,we'llanalyzetheschedulingbehavioroftwo loggingapproaches. Mostloggingframeworksarethinwrappersaroundprintln;whenyouhavesomethingtolog,justwriteitoutright then and there. Another approach was shown in LogWriter on page 152: the logging is performed in a dedicated backgroundthreadinsteadofbytherequestingthread.Fromthedeveloper'sperspective,bothapproachesareroughly equivalent. But there may be a difference in performance, depending on the volume of logging activity, how many threadsaredoinglogging,andotherfactorssuchasthecostofcontextswitching.[16] [16]BuildingaloggerthatmovestheI/Otoanotherthreadmayimproveperformance,butitalsointroducesanumberofdesigncomplications, suchasinterruption(whathappensifathreadblockedinaloggingoperationisinterrupted?),serviceguarantees(doestheloggerguaranteethata successfullyqueuedlogmessagewillbeloggedpriortoserviceshutdown?),saturationpolicy(whathappenswhentheproducerslogmessages fasterthantheloggerthreadcanhandlethem?),andservicelifecycle(howdoweshutdownthelogger,andhowdowecommunicatetheservice statetoproducers?). TheservicetimeforaloggingoperationincludeswhatevercomputationisassociatedwiththeI/Ostreamclasses;ifthe I/Ooperationblocks,italsoincludesthedurationforwhichthethreadisblocked.Theoperatingsystemwilldeschedule theblockedthreaduntiltheI/Ocompletes,andprobablyalittlelonger.WhentheI/Ocompletes,otherthreadsare probablyactiveandwillbeallowedtofinishouttheirschedulingquanta,andthreadsmayalreadybewaitingaheadof usontheschedulingqueueͲfurtheraddingtoservicetime.Alternatively,ifmultiplethreadsareloggingsimultaneously, theremaybecontentionfortheoutputstreamlock,inwhichcasetheresultisthesameaswithblockingI/OͲthe thread blocks waiting for the lock and gets switched out. Inline logging involves I/O and locking, which can lead to increasedcontextswitchingandthereforeincreasedservicetimes. Increasingrequestservicetimeisundesirableforseveralreasons.First,servicetimeaffectsqualityofservice:longer servicetimesmeansomeoneiswaitinglongerforaresult.Butmoresignificantly,longerservicetimesinthiscasemean morelockcontention.The"getin,getout"principleofSection11.4.1tellsusthatweshouldholdlocksasbrieflyas possible,becausethelongeralockisheld,themorelikelythatlockwillbecontended.IfathreadblockswaitingforI/O while holding a lock, another thread is more likely to want the lock while the first thread is holding it. Concurrent systemsperformmuchbetterwhenmostlockacquisitionsareuncontended,becausecontendedlockacquisitionmeans morecontextswitches.Acodingstylethatencouragesmorecontextswitchesthusyieldsloweroverallthroughput. MovingtheI/OoutoftherequestͲprocessingthreadislikelytoshortenthemeanservicetimeforrequestprocessing. ThreadscallinglognolongerblockwaitingfortheoutputstreamlockorforI/Otocomplete;theyneedonlyqueuethe messageandcanthenreturntotheirtask.Ontheotherhand,we'veintroducedthepossibilityofcontentionforthe messagequeue,buttheputoperationislighterͲweightthantheloggingI/O(whichmightrequiresystemcalls)andsois lesslikelytoblockinactualuse(aslongasthequeueisnotfull).Becausetherequestthreadisnowlesslikelytoblock,it is less likely to be contextͲswitched out in the middle of a request. What we've done is turned a complicated and uncertaincodepathinvolvingI/OandpossiblelockcontentionintoastraightͲlinecodepath. Tosomeextent,wearejustmovingtheworkaround,movingtheI/Otoathreadwhereitscostisn'tperceivedbythe user(whichmayinitselfbeawin).ButbymovingalltheloggingI/Otoasinglethread,wealsoeliminatethechanceof contentionfortheoutputstreamandthuseliminateasourceofblocking.Thisimprovesoverallthroughputbecause fewerresourcesareconsumedinscheduling,contextswitching,andlockmanagement. MovingtheI/OfrommanyrequestͲprocessingthreadstoasingleloggerthreadissimilartothedifferencebetweena bucket brigade and a collection of individuals fighting a fire. In the "hundred guys running around with buckets" approach,youhaveagreaterchanceofcontentionatthewatersourceandatthefire(resultinginoveralllesswater  152 JavaConcurrencyInPractice deliveredtothefire),plusgreaterinefficiencybecauseeachworkeriscontinuouslyswitchingmodes(filling,running, dumping,running,etc.).InthebucketͲbrigadeapproach,theflowofwaterfromthesourcetotheburningbuildingis constant, less energy is expended transporting the water to the fire, and each worker focuses on doing one job continuously.JustasinterruptionsaredisruptiveandproductivityͲreducingtohumans,blockingandcontextswitching aredisruptivetothreads. Summary Because one of the most common reasons to use threads is to exploit multiple processors, in discussing the performanceofconcurrentapplications,weareusuallymoreconcernedwiththroughputorscalabilitythanwearewith rawservicetime.Amdahl'slawtellsusthatthescalabilityofanapplicationisdrivenbytheproportionofcodethatmust beexecutedserially.SincetheprimarysourceofserializationinJavaprogramsistheexclusiveresourcelock,scalability canoftenbeimprovedbyspendinglesstimeholdinglocks,eitherbyreducinglockgranularity,reducingthedurationfor whichlocksareheld,orreplacingexclusivelockswithnonexclusiveornonͲblockingalternatives.   1536BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms Chapter12.TestingConcurrentPrograms Concurrent programs employ similar design principles and patterns to sequential programs. The difference is that concurrentprogramshaveadegreeofnonͲdeterminismthatsequentialprogramsdonot,increasingthenumberof potentialinteractionsandfailuremodesthatmustbeplannedforandanalyzed. Similarly,testingconcurrentprogramsusesandextendsideasfromtestingsequentialones.Thesametechniquesfor testing correctness and performance in sequential programs can be applied to concurrent programs, but with concurrentprogramsthespaceofthingsthatcangowrongismuchlarger.Themajorchallengeinconstructingtestsfor concurrent programs is that potential failures may be rare probabilistic occurrences rather than deterministic ones; teststhatdisclosesuchfailuresmustbemoreextensiveandrunforlongerthantypicalsequentialtests. Mosttestsofconcurrentclassesfallintooneorbothoftheclassiccategoriesofsafetyandliveness.InChapter1,we definedsafetyas"nothingbadeverhappens"andlivenessas"somethinggoodeventuallyhappens". Tests of safety, which verify that a class's behavior conforms to its specification, usually take the form of testing invariants.Forexample,inalinkedlistimplementationthatcachesthesizeofthelisteverytimeitismodified,one safetytestwouldbetocomparethecachedcountagainsttheactualnumberofelementsinthelist.InasingleͲthreaded programthisiseasy,sincethelist contentsdonotchangewhileyouaretestingitsproperties.Butinaconcurrent program,suchatestislikelytobefraughtwithracesunlessyoucanobservethecountfieldandcounttheelementsina single atomic operation. This can be done by locking the list for exclusive access, employing some sort of "atomic snapshot"featureprovidedbytheimplementation,orbyusing"testpoints"providedbytheimplementationthatlet testsassertinvariantsorexecutetestcodeatomically. In this book, we've used timing diagrams to depict "unlucky" interactions that could cause failures in incorrectly constructedclasses;testprogramsattempttosearchenoughofthestatespacethatsuchbadluckeventuallyoccurs. Unfortunately, test code can introduce timing or synchronization artifacts that can mask bugs that might otherwise manifestthemselves.[1] [1]BugsthatdisappearwhenyouadddebuggingortestcodeareplayfullycalledHeisenbugs. Liveness properties present their own testing challenges. Liveness tests include tests of progress and nonͲprogress, whicharehardtoquantifyͲhowdoyouverifythatamethodisblockingandnotmerelyrunningslowly?Similarly,how doyoutestthatanalgorithmdoesnotdeadlock?Howlongshouldyouwaitbeforeyoudeclareittohavefailed? Relatedtolivenesstestsareperformancetests.Performancecanbemeasuredinanumberofways,including: Throughput:therateatwhichasetofconcurrenttasksiscompleted; Responsiveness:thedelaybetweenarequestforandcompletionofsomeaction(alsocalledlatency);or Scalability:theimprovementinthroughput(orlackthereof)asmoreresources(usuallyCPUs)aremadeavailable. 12.1.TestingforCorrectness DevelopingunittestsforaconcurrentclassstartswiththesameanalysisasforasequentialclassͲidentifyinginvariants and postͲconditions that are amenable to mechanical checking. If you are lucky, many of these are present in the specification;therestofthetime,writingtestsisanadventureiniterativespecificationdiscovery. As a concrete illustration, we're going to build a set of test cases for a bounded buffer. Listing 12.1 shows our BoundedBufferimplementation,usingSemaphoretoimplementtherequiredboundingandblocking. BoundedBufferimplementsafixedͲlengtharrayͲbasedqueuewithblockingputandtakemethodscontrolledbyapair ofcountingsemaphores.The availableItemssemaphorerepresentsthenumberofelementsthatcanberemoved fromthebuffer,andisinitiallyzero(sincethebufferisinitiallyempty).Similarly, availableSpacesrepresentshow manyitemscanbeinsertedintothebuffer,andisinitializedtothesizeofthebuffer. AtakeoperationfirstrequiresthatapermitbeobtainedfromavailableItems.Thissucceedsimmediatelyifthebuffer isnonempty,andotherwiseblocksuntilthebufferbecomesnonempty.Onceapermitisobtained,thenextelement fromthebufferisremovedandapermitisreleasedtotheavailableSpacessemaphore.[2]Theputoperationisdefined conversely,sothatonexitfromeithertheputortakemethods,thesumofthecountsofbothsemaphoresalways equals the bound. (In practice, if you need a bounded buffer you should use ArrayBlockingQueue or LinkedBlockingQueue rather than rolling your own, but the technique used here illustrates how insertions and removalscanbecontrolledinotherdatastructuresaswell.)  154 JavaConcurrencyInPractice [2]Inacountingsemaphore,thepermitsarenotrepresentedexplicitlyorassociatedwithanowningthread;areleaseoperationcreatesa permitandanacquireoperationconsumesone. Listing12.1.BoundedBufferUsingSemaphore. @ThreadSafe public class BoundedBuffer { private final Semaphore availableItems, availableSpaces; @GuardedBy("this") private final E[] items; @GuardedBy("this") private int putPosition = 0, takePosition = 0; public BoundedBuffer(int capacity) { availableItems = new Semaphore(0); availableSpaces = new Semaphore(capacity); items = (E[]) new Object[capacity]; } public boolean isEmpty() { return availableItems.availablePermits() == 0; } public boolean isFull() { return availableSpaces.availablePermits() == 0; } public void put(E x) throws InterruptedException { availableSpaces.acquire(); doInsert(x); availableItems.release(); } public E take() throws InterruptedException { availableItems.acquire(); E item = doExtract(); availableSpaces.release(); return item; } private synchronized void doInsert(E x) { int i = putPosition; items[i] = x; putPosition = (++i == items.length)? 0 : i; } private synchronized E doExtract() { int i = takePosition; E x = items[i]; items[i] = null; takePosition = (++i == items.length)? 0 : i; return x; } } 12.1.1.BasicUnitTests Themostbasicunittestsfor BoundedBufferaresimilartowhatwe'duseinasequentialcontextcreateabounded buffer,callitsmethods,andassertpostͲconditionsandinvariants.Someinvariantsthatquicklycometomindarethata freshlycreatedbuffershouldidentifyitselfasempty,andalsoasnotfull.Asimilarbutslightlymorecomplicatedsafety testistoinsertNelementsintoabufferwithcapacityN(whichshouldsucceedwithoutblocking),andtestthatthe bufferrecognizesthatitisfull(andnotempty).JUnittestmethodsforthesepropertiesareshowninListing12.2. Listing12.2.BasicUnitTestsforBoundedBuffer. class BoundedBufferTest extends TestCase { void testIsEmptyWhenConstructed() { BoundedBuffer bb = new BoundedBuffer(10); assertTrue(bb.isEmpty()); assertFalse(bb.isFull()); } void testIsFullAfterPuts() throws InterruptedException { BoundedBuffer bb = new BoundedBuffer(10); for (int i = 0; i < 10; i++) bb.put(i); assertTrue(bb.isFull()); assertFalse(bb.isEmpty()); } } Thesesimpletestmethodsareentirelysequential.Includingasetofsequentialtestsinyourtestsuiteisoftenhelpful, sincetheycandisclosewhenaproblemisnotrelatedtoconcurrencyissuesbeforeyoustartlookingfordataraces.  1556BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 12.1.2.TestingBlockingOperations Testsofessentialconcurrencypropertiesrequireintroducingmorethanonethread.Mosttestingframeworksarenot veryconcurrencyͲfriendly:theyrarelyincludefacilitiestocreatethreadsormonitorthemtoensurethattheydonotdie unexpectedly.Ifahelperthreadcreatedbyatestcasediscoversafailure,theframeworkusuallydoesnotknowwith whichtestthethreadisassociated,sosomeworkmayberequiredtorelaysuccessorfailureinformationbacktothe maintestrunnerthreadsoitcanbereported. For the conformance tests for java.util.concurrent, it was important that failures be clearly associated with a specifictest.HencetheJSR166ExpertGroupcreatedabaseclass[3]thatprovidedmethodstorelayandreportfailures duringtearDown,followingtheconventionthateverytestmustwaituntilallthethreadsitcreatedterminate.Youmay notneedtogotosuchlengths;thekeyrequirementsarethatitbeclearwhetherthetestspassedandthatfailure informationisreportedsomewhereforuseindiagnosingtheproblem. [3]http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/tck/JSR166TestCase.java Ifamethod issupposed toblockundercertainconditions,thenatestforthatbehaviorshouldsucceedonlyifthe threaddoesnotproceed.Testingthatamethodblocksissimilartotestingthatamethodthrowsanexception;ifthe methodreturnsnormally,thetesthasfailed. Testingthatamethodblocksintroducesanadditionalcomplication:oncethemethodsuccessfullyblocks,youhaveto convinceitsomehowtounblock.Theobviouswaytodothisisviainterruptionstartablockingactivityinaseparate thread,waituntilthethreadblocks,interruptit,andthenassertthattheblockingoperationcompleted.Ofcourse,this requiresyourblockingmethodstorespondtointerruptionbyreturningearlyorthrowingInterruptedException. The"waituntilthethreadblocks"partiseasiersaidthandone;inpractice,youhavetomakeanarbitrarydecision about how long the few instructions being executed could possibly take, and wait longer than that. You should be preparedtoincreasethisvalueifyouarewrong(inwhichcaseyouwillseespurioustestfailures). Listing12.3showsanapproachtotestingblocking operations.Itcreatesa"taker"threadthatattemptsto take an elementfromanemptybuffer.Iftakesucceeds,itregistersfailure.Thetestrunnerthreadstartsthetakerthread,waits a long time, and then interrupts it. If the taker thread has correctly blocked in the take operation, it will throw InterruptedException,andthecatchblockforthisexceptiontreatsthisassuccessandletsthethreadexit.Themain testrunnerthreadthenattemptstojoinwiththetakerthreadandverifiesthatthejoinreturnedsuccessfullybycalling Thread.isAlive;ifthetakerthreadrespondedtotheinterrupt,thejoinshouldcompletequickly. Thetimedjoinensuresthatthetestcompleteseveniftakegetsstuckinsomeunexpectedway.Thistestmethodtests severalpropertiesoftakenotonlythatitblocksbutthat,wheninterrupted,itthrowsInterruptedException.Thisis oneofthefewcasesinwhichitisappropriatetosubclassThreadexplicitlyinsteadofusingaRunnableinapool:in ordertotestproperterminationwithjoin.Thesameapproachcanbeusedtotestthatthetakerthreadunblocksafter anelementisplacedinthequeuebythemainthread. It is tempting to use Thread.getState to verify that the thread is actually blocked on a condition wait, but this approachisnotreliable.ThereisnothingthatrequiresablockedthreadevertoentertheWAITINGorTIMED_WAITING states,sincetheJVMcanchoosetoimplementblockingbyspinͲwaitinginstead.Similarly,becausespuriouswakeups from Object.waitor Condition.awaitarepermitted(seeChapter14),athreadinthe WAITINGor TIMED_WAITING statemaytemporarilytransitiontoRUNNABLEeveniftheconditionforwhichitiswaitingisnotyettrue.Evenignoring theseimplementationoptions,itmaytakesometimeforthetargetthreadtosettleintoablockingstate.Theresultof Thread.getStateshouldnotbeusedforconcurrencycontrol,andisoflimitedusefulnessfortestingͲitsprimaryutility isasasourceofdebugginginformation.  156 JavaConcurrencyInPractice Listing12.3.TestingBlockingandResponsivenesstoInterruption. void testTakeBlocksWhenEmpty() { final BoundedBuffer bb = new BoundedBuffer(10); Thread taker = new Thread() { public void run() { try { int unused = bb.take(); fail(); // if we get here, it's an error } catch (InterruptedException success) { } }}; try { taker.start(); Thread.sleep(LOCKUP_DETECT_TIMEOUT); taker.interrupt(); taker.join(LOCKUP_DETECT_TIMEOUT); assertFalse(taker.isAlive()); } catch (Exception unexpected) { fail(); } } 12.1.3.TestingSafety ThetestsinListings12.2and12.3testimportantpropertiesoftheboundedbuffer,butareunlikelytodiscloseerrors stemmingfromdataraces.Totestthataconcurrentclassperformscorrectlyunderunpredictableconcurrentaccess,we needtosetupmultiplethreadsperformingputandtakeoperationsoversomeamountoftimeandthensomehowtest thatnothingwentwrong. Constructing tests to disclose safety errors in concurrent classes is a chickenͲandͲegg problem: the test programs themselves are concurrent programs. Developing good concurrent tests can be more difficult than developing the classestheytest. Thechallengetoconstructingeffectivesafetytestsforconcurrentclassesisidentifyingeasilycheckedpropertiesthat will,withhighprobability,failifsomethinggoeswrong,whileatthesametimenotlettingthefailureͲauditingcodelimit concurrencyartificially.Itisbestifcheckingthetestpropertydoesnotrequireanysynchronization. OneapproachthatworkswellwithclassesusedinproducerͲconsumerdesigns(likeBoundedBuffer)istocheckthat everything put into a queue or buffer comes out of it, and that nothing else does. A naive implementation of this approachwouldinserttheelementintoa"shadow"listwhenitisputonthequeue,removeitfromthelistwhenitis removedfromthequeue,andassertthattheshadowlistisemptywhenthetesthasfinished.Butthisapproachwould distorttheschedulingofthetestthreadsbecausemodifyingtheshadowlistwouldrequiresynchronizationandpossibly blocking. AbetterapproachistocomputechecksumsoftheelementsthatareenqueuedanddequeuedusinganorderͲsensitive checksumfunction,andcomparethem.Iftheymatch,thetestpasses.Thisapproachworksbestwhenthereisasingle producerputtingelementsintothebufferandasingleconsumertakingthemout,becauseitcantestnotonlythatthe rightelements(probably)cameoutbutthattheycameoutintherightorder. ExtendingthisapproachtoamultipleͲproducer,multipleͲconsumersituationrequiresusingachecksumfunctionthatis insensitivetotheorderinwhichtheelementsarecombined,sothatmultiplechecksumscanbecombinedafterthe test.Otherwise,synchronizingaccesstoasharedchecksumfieldcouldbecomeaconcurrencybottleneckordistortthe timingofthetest.(Anycommutativeoperation,suchasadditionorXOR,meetstheserequirements.) Toensurethatyourtestactuallytestswhatyouthinkitdoes,itisimportantthatthechecksumsthemselvesnotbe guessablebythecompiler.Itwouldbeabadideatouseconsecutiveintegersasyourtestdatabecausethentheresult wouldalwaysbethesame,andasmartcompilercouldconceivablyjustprecomputeit. Toavoidthisproblem,testdatashouldbegeneratedrandomly,butmanyotherwiseeffectivetestsarecompromisedby apoorchoiceofrandomnumbergenerator(RNG).Randomnumbergenerationcancreatecouplingsbetweenclasses andtimingartifactsbecausemostrandomnumbergeneratorclassesarethreadsafeandthereforeintroduceadditional synchronization.[4]GivingeachthreaditsownRNGallowsanonͲthreadͲsafeRNGtobeused. [4]Manybenchmarksare,unbeknownsttotheirdevelopersorusers,simplytestsofhowgreataconcurrencybottlenecktheRNGis. Rather than usingageneralͲpurposeRNG,itisbettertousesimplepseudorandomfunctions.Youdon'tneedhighͲ qualityrandomness;allyouneedisenoughrandomnesstoensurethenumberschangefromruntorun.Thexor-Shift functioninListing12.4(Marsaglia,2003)isamongthecheapestmediumͲqualityrandomnumberfunctions.Startingit  1576BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms offwithvaluesbasedonhashCodeandnanoTimemakesthesumsbothunguessableandalmostalwaysdifferentfor eachrun. Listing12.4.MediumǦqualityRandomNumberGeneratorSuitableforTesting. static int xorShift(int y) { y ^= (y << 6); y ^= (y >>> 21); y ^= (y << 7); return y; } PutTakeTestinListings12.5and12.6startsNproducerthreadsthatgenerateelementsandenqueuethem,andN consumerthreadsthatdequeuethem.Eachthreadupdatesthechecksumoftheelementsastheygoinorout,usinga perthreadchecksumthatiscombinedattheendofthetestrunsoastoaddnomoresynchronizationorcontention thanrequiredtotestthebuffer. Dependingonyourplatform,creatingandstartingathreadcanbeamoderatelyheavyweightoperation.Ifyourthread isshortͲrunningandyoustartanumberofthreadsinaloop,thethreadsrunsequentiallyratherthanconcurrentlyin theworstcase.EveninthenotͲquiteͲworstcase,thefactthatthefirstthreadhasaheadstartontheothersmeansthat youmaygetfewerinterleavingsthanexpected:thefirstthreadrunsbyitselfforsomeamountoftime,andthenthe first two threads run concurrently for some amount of time, and only eventually are all the threads running concurrently.(Thesamethinghappensattheendoftherun:thethreadsthatgotaheadstartalsofinishearly.) WepresentedatechniqueformitigatingthisprobleminSection5.5.1,usingaCountDownLatchasastartinggateand anotherasafinishgate.AnotherwaytogetthesameeffectistouseaCyclicBarrier,initializedwiththenumberof workerthreadsplusone,andhavetheworkerthreadsandthetestdriverwaitatthebarrieratthebeginningandendof theirrun.Thisensuresthatallthreadsareupandrunningbeforeanystartworking.PutTakeTestusesthistechniqueto coordinatestartingandstoppingtheworkerthreads,creatingmorepotentialconcurrentinterleavings.Westillcan't guarantee that the scheduler won't run each thread to completion sequentially, but making the runs long enough reducestheextenttowhichschedulingdistortsourresults. The final trick employed by PutTakeTest is to use a deterministic termination criterion so that no additional interͲ threadcoordinationisneededtofigureoutwhenthetestisfinished.Thetestmethodstartsexactlyasmanyproducers asconsumersandeachofthemputsortakesthesamenumberofelements,sothetotalnumberofitemsaddedand removedisthesame. Testslike PutTakeTesttend tobe goodatfindingsafetyviolations.Forexample,a commonerrorinimplementing semaphoreͲcontrolled buffers is to forget that the code actually doing the insertion and extraction requires mutual exclusion(usingsynchronizedorReentrantLock).AsamplerunofPutTakeTestwithaversionofBoundedBufferthat omits making doInsert and doExtract synchronized fails fairly quickly. Running PutTakeTest with a few dozen threadsiteratingafewmilliontimesonbuffersofvariouscapacityonvarioussystemsincreasesourconfidenceabout thelackofdatacorruptioninputandtake. Tests should be run on multiprocessor systems to increase the diversity of potential interleavings. However, having morethanafewCPUsdoesnotnecessarilymaketestsmoreeffective.TomaximizethechanceofdetectingtimingͲ sensitivedataraces,thereshouldbemoreactivethreadsthanCPUs,sothatatanygiventimesomethreadsarerunning andsomeareswitchedout,thusreducingthepredictabilityofinteractionsbetweenthreads.  158 JavaConcurrencyInPractice Listing12.5.ProducerǦconsumerTestProgramforBoundedBuffer. public class PutTakeTest { private static final ExecutorService pool = Executors.newCachedThreadPool(); private final AtomicInteger putSum = new AtomicInteger(0); private final AtomicInteger takeSum = new AtomicInteger(0); private final CyclicBarrier barrier; private final BoundedBuffer bb; private final int nTrials, nPairs; public static void main(String[] args) { new PutTakeTest(10, 10, 100000).test(); // sample parameters pool.shutdown(); } PutTakeTest(int capacity, int npairs, int ntrials) { this.bb = new BoundedBuffer(capacity); this.nTrials = ntrials; this.nPairs = npairs; this.barrier = new CyclicBarrier(npairs* 2 + 1); } void test() { try { for (int i = 0; i < nPairs; i++) { pool.execute(new Producer()); pool.execute(new Consumer()); } barrier.await(); // wait for all threads to be ready barrier.await(); // wait for all threads to finish assertEquals(putSum.get(), takeSum.get()); } catch (Exception e) { throw new RuntimeException(e); } } class Producer implements Runnable { /* Listing 12.6 */ } class Consumer implements Runnable { /* Listing 12.6 */ } } Listing12.6.ProducerandConsumerClassesUsedinPutTakeTest. /* inner classes of PutTakeTest (Listing 12.5) */ class Producer implements Runnable { public void run() { try { int seed = (this.hashCode() ^ (int)System.nanoTime()); int sum = 0; barrier.await(); for (int i = nTrials; i > 0; --i) { bb.put(seed); sum += seed; seed = xorShift(seed); } putSum.getAndAdd(sum); barrier.await(); } catch (Exception e) { throw new RuntimeException(e); } } } class Consumer implements Runnable { public void run() { try { barrier.await(); int sum = 0; for (int i = nTrials; i > 0; --i) { sum += bb.take(); } takeSum.getAndAdd(sum); barrier.await(); } catch (Exception e) { throw new RuntimeException(e); } } }   1596BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms Inteststhatrununtiltheycompleteafixednumberofoperations,itispossiblethatthetestcasewillneverfinishifthe codebeingtestedencountersanexceptionduetoabug.Themostcommonwaytohandlethisistohavethetest frameworkabortteststhatdonotterminatewithinacertainamountoftime;howlongtowaitshouldbedetermined empirically,andfailuresmustthenbeanalyzedtoensurethattheproblemwasn'tjustthatyoudidn'twaitlongenough. (Thisproblemisnotuniquetotestingconcurrentclasses;sequentialtestsmustalsodistinguishbetweenlongͲrunning andinfiniteloops.) 12.1.4.TestingResourceManagement Thetestssofarhavebeenconcernedwithaclass'sadherencetoitsspecificationͲthatitdoeswhatitissupposedtodo. Asecondaryaspecttotestisthatitdoesnotdothingsitisnotsupposedtodo,suchasleakresources.Anyobjectthat holdsormanagesotherobjectsshouldnotcontinuetomaintainreferencestothoseobjectslongerthannecessary.Such storage leaks prevent garbage collectors from reclaiming memory (or threads, file handles, sockets, database connections,orotherlimitedresources)andcanleadtoresourceexhaustionandapplicationfailure. ResourcemanagementissuesareespeciallyimportantforclasseslikeBoundedBuffertheentirereasonforboundinga bufferistopreventapplicationfailureduetoresourceexhaustionwhenproducersgettoofaraheadofconsumers. Boundingcausesoverlyproductiveproducerstoblockratherthancontinuetocreateworkthatwillconsumemoreand morememoryorotherresources. UndesirablememoryretentioncanbeeasilytestedwithheapͲinspectiontoolsthatmeasureapplicationmemoryusage; avarietyofcommercialandopenͲsourceheapͲprofilingtoolscandothis.ThetestLeakmethodinListing12.7contains placeholdersforaheapͲinspection tooltosnapshotthe heap,whichforcesagarbage collection[5]and thenrecords informationabouttheheapsizeandmemoryusage. [5]Technically,itisimpossibletoforceagarbagecollection;System.gconlysuggeststotheJVMthatthismightbeagoodtimetoperforma garbagecollection.HotSpotcanbeinstructedtoignoreSystem.gccallswith-XX:+DisableExplicitGC. ThetestLeakmethodinsertsseverallargeobjectsintoaboundedbufferandthenremovesthem;memoryusageat heapsnapshot#2shouldbeapproximatelythesameasatheapsnapshot#1.Ontheotherhand,ifdoExtractforgotto nulloutthereferencetothereturnedelement(items[i]=null),thereportedmemoryusageatthetwosnapshots woulddefinitelynotbethesame.(Thisisoneofthefewtimeswhereexplicitnullingisnecessary;mostofthetime,itis eithernothelpfuloractuallyharmful[EJItem5].) 12.1.5.UsingCallbacks CallbackstoclientͲprovidedcodecanbehelpfulinconstructingtestcases;callbacksareoftenmadeatknownpointsin anobject'slifecyclethataregoodopportunitiestoassertinvariants.Forexample,ThreadPoolExecutormakescallsto thetaskRunnablesandtotheThreadFactory. Listing12.7.TestingforResourceLeaks. class Big { double[] data = new double[100000]; } void testLeak() throws InterruptedException { BoundedBuffer bb = new BoundedBuffer(CAPACITY); int heapSize1 = /* snapshot heap */ ; for (int i = 0; i < CAPACITY; i++) bb.put(new Big()); for (int i = 0; i < CAPACITY; i++) bb.take(); int heapSize2 = /* snapshot heap */ ; assertTrue(Math.abs(heapSize1-heapSize2) < THRESHOLD); } Testingathreadpoolinvolvestestinganumberofelementsofexecutionpolicy:thatadditionalthreadsarecreated when they are supposed to, but not when they are not supposed to; that idle threads get reaped when they are supposedto,etc.Constructingacomprehensivetestsuitethatcoversallthepossibilitiesisamajoreffort,butmanyof themcanbetestedfairlysimplyindividually. Wecaninstrumentthreadcreationbyusingacustomthreadfactory.TestingThreadFactoryinListing12.8maintainsa count of created threads; test cases can then verify the number of threads created during a test run. TestingThreadFactorycouldbeextendedtoreturnacustomThreadthatalsorecordswhenthethreadterminates,so thattestcasescanverifythatthreadsarereapedinaccordancewiththeexecutionpolicy.  160 JavaConcurrencyInPractice Listing12.8.ThreadFactoryforTestingThreadPoolExecutor. class TestingThreadFactory implements ThreadFactory { public final AtomicInteger numCreated = new AtomicInteger(); private final ThreadFactory factory = Executors.defaultThreadFactory(); public Thread newThread(Runnable r) { numCreated.incrementAndGet(); return factory.newThread(r); } } Ifthecorepoolsizeissmallerthanthemaximumsize,thethreadpoolshouldgrowasdemandforexecutionincreases. SubmittinglongͲrunningtaskstothepoolmakesthenumberofexecutingtasksstayconstantforlongenoughtomakea fewassertions,suchastestingthatthepoolisexpandedasexpected,asshowninListing12.9. Listing12.9.TestMethodtoVerifyThreadPoolExpansion. public void testPoolExpansion() throws InterruptedException { int MAX_SIZE = 10; ExecutorService exec = Executors.newFixedThreadPool(MAX_SIZE); for (int i = 0; i < 10* MAX_SIZE; i++) exec.execute(new Runnable() { public void run() { try { Thread.sleep(Long.MAX_VALUE); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } }); for (int i = 0; i < 20 && threadFactory.numCreated.get() < MAX_SIZE; i++) Thread.sleep(100); assertEquals(threadFactory.numCreated.get(), MAX_SIZE); exec.shutdownNow(); } 12.1.6.GeneratingMoreInterleavings SincemanyofthepotentialfailuresinconcurrentcodearelowͲprobabilityevents,testingforconcurrencyerrorsisa numbersgame,buttherearesomethingsyoucandotoimproveyourchances.We'vealreadymentionedhowrunning onmultiprocessorsystemswithfewerprocessorsthanactivethreadscangeneratemoreinterleavingsthaneithera singleͲprocessorsystemoronewithmanyprocessors.Similarly,testingonavarietyofsystemswithdifferentprocessor counts,operatingsystems,andprocessorarchitecturescandiscloseproblemsthatmightnotoccuronallsystems. Ausefultrickforincreasingthenumberofinterleavings,andthereforemoreeffectivelyexploringthestatespaceof yourprograms,istouseThread.yieldtoencouragemorecontextswitchesduringoperationsthataccesssharedstate. (TheeffectivenessofthistechniqueisplatformͲspecific,sincetheJVMisfreetotreatThread.yieldasanoͲop[JLS 17.9];usingashortbutnonzerosleepwouldbeslowerbutmorereliable.)ThemethodinListing12.10transferscredits fromoneaccounttoanother;betweenthetwoupdateoperations,invariantssuchas"sumofallaccountsequalszero" donothold.Bysometimesyieldinginthemiddleofanoperation,youmayactivatetimingͲsensitivebugsincodethat doesnotuseadequatesynchronizationtoaccessstate.Theinconvenienceofaddingthesecallsfortestingandremoving themforproductioncanbereducedbyaddingthemusingaspectͲorientedprogramming(AOP)tools. Listing12.10.UsingThread.yieldtoGenerateMoreInterleavings. public synchronized void transferCredits(Account from, Account to, int amount) { from.setBalance(from.getBalance() - amount); if (random.nextInt(1000) > THRESHOLD) Thread.yield(); to.setBalance(to.getBalance() + amount); } 12.2.TestingforPerformance Performancetestsareoftenextendedversionsoffunctionalitytests.Infact,itisalmostalwaysworthwhiletoinclude somebasicfunctionalitytestingwithinperformanceteststoensurethatyouarenottestingtheperformanceofbroken code.  1616BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms Whilethereisdefinitelyoverlapbetweenperformanceandfunctionalitytests,theyhavedifferentgoals.Performance testsseektomeasureendͲtoͲendperformancemetricsforrepresentativeusecases.Pickingareasonablesetofusage scenarios is not always easy; ideally, tests should reflect how the objects being tested are actually used in your application. Insomecasesanappropriatetestscenarioisobvious.BoundedbuffersarenearlyalwaysusedinproducerͲconsumer designs,soitissensibletomeasurethethroughputofproducersfeedingdatatoconsumers.Wecaneasilyextend PutTakeTesttobecomeaperformancetestforthisscenario. Acommonsecondarygoalofperformancetestingistoselectsizingsempiricallyforvariousboundsnumbersofthreads, buffercapacities,andsoon.Whilethesevaluesmightturnouttobesensitiveenoughtoplatformcharacteristics(such asprocessortypeorevenprocessorsteppinglevel,numberofCPUs,ormemorysize)torequiredynamicconfiguration, itisequallycommonthatreasonablechoicesforthesevaluesworkwellacrossawiderangeofsystems. 12.2.1.ExtendingPutTakeTesttoAddTiming TheprimaryextensionwehavetomaketoPutTakeTestistomeasurethetimetakenforarun.Ratherthanattempting tomeasurethetimeforasingleoperation,wegetamoreaccuratemeasurebytimingtheentirerunanddividingbythe numberofoperationstogetaperͲoperationtime.WearealreadyusingaCyclicBarriertostartandstoptheworker threads,sowecanextendthisbyusingabarrieractionthatmeasuresthestartandendtime,asshowninListing12.11. WecanmodifytheinitializationofthebarriertousethisbarrieractionbyusingtheconstructorforCyclicBarrierthat acceptsabarrieraction: Listing12.11.BarrierǦbasedTimer. this.timer = new BarrierTimer(); this.barrier = new CyclicBarrier(npairs * 2 + 1, timer); public class BarrierTimer implements Runnable { private boolean started; private long startTime, endTime; public synchronized void run() { long t = System.nanoTime(); if (!started) { started = true; startTime = t; } else endTime = t; } public synchronized void clear() { started = false; } public synchronized long getTime() { return endTime - startTime; } } ThemodifiedtestmethodusingthebarrierͲbasedtimerisshowninListing12.12. WecanlearnseveralthingsfromrunningTimedPutTakeTest.OneisthethroughputoftheproducerͲconsumerhandoff operationforvariouscombinationsofparameters;anotherishowtheboundedbufferscaleswithdifferentnumbersof threads;athirdishowwemightselecttheboundsize.Answeringthesequestionsrequiresrunningthetestforvarious combinationsofparameters,sowe'llneedamaintestdriver,showninListing12.13. Figure12.1showssomesampleresultsona4Ͳwaymachine,usingbuffercapacitiesof1,10,100,and1000.Wesee immediatelythatabuffersizeofonecausesverypoorthroughput;thisisbecauseeachthreadcanmakeonlyatinybit of progress before blocking and waiting for another thread. Increasing buffer size to ten helps dramatically, but increasespasttenofferdiminishingreturns.  162 JavaConcurrencyInPractice Figure12.1.TimedPutTakeTestwithVariousBufferCapacities.   Itmaybesomewhatpuzzlingatfirstthataddingalotmorethreadsdegradesperformanceonlyslightly.Thereasonis hardtoseefromthedata,buteasytoseeonaCPUperformancemetersuchasperfbarwhilethetestisrunning:even withmanythreads,notmuchcomputationisgoingon,andmostofitisspentblockingandunblockingthreads.Sothere isplentyofCPUslackformorethreadstodothesamethingwithouthurtingperformanceverymuch. However,becarefulaboutconcludingfromthisdatathatyoucanalwaysaddmorethreadstoaproducerͲconsumer programthatusesaboundedbuffer.Thistestisfairlyartificialinhowitsimulatestheapplication;theproducersdo almost no work to generate the item placed on the queue, and the consumers do almost no work with the item retrieved. If the worker threads in a real producerͲconsumer application do some nontrivial work to produce and consumeitems(asisgenerallythecase),thenthisslackwoulddisappearandtheeffectsofhavingtoomanythreads could be very noticeable. The primary purpose of this test is to measure what constraints the producerͲconsumer handoffviatheboundedbufferimposesonoverallthroughput. Listing12.12.TestingwithaBarrierǦbasedTimer. public void test() { try { timer.clear(); for (int i = 0; i < nPairs; i++) { pool.execute(new Producer()); pool.execute(new Consumer()); } barrier.await(); barrier.await(); long nsPerItem = timer.getTime() / (nPairs* (long)nTrials); System.out.print("Throughput: " + nsPerItem + " ns/item"); assertEquals(putSum.get(), takeSum.get()); } catch (Exception e) { throw new RuntimeException(e); } }   1636BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms Listing12.13.DriverProgramǦforTimedPutTakeTest. public static void main(String[] args) throws Exception { int tpt = 100000; // trials per thread for (int cap = 1; cap <= 1000; cap*= 10) { System.out.println("Capacity: " + cap); for (int pairs = 1; pairs <= 128; pairs*= 2) { TimedPutTakeTest t = new TimedPutTakeTest(cap, pairs, tpt); System.out.print("Pairs: " + pairs + "\t"); t.test(); System.out.print("\t"); Thread.sleep(1000); t.test(); System.out.println(); Thread.sleep(1000); } } pool.shutdown(); } 12.2.2.ComparingMultipleAlgorithms WhileBoundedBufferisafairlysolidimplementationthatperformsreasonablywell,itturnsouttobenomatchfor either ArrayBlockingQueue or LinkedBlockingQueue (which explains why this buffer algorithm wasn't selected for inclusionintheclasslibrary).Thejava.util.concurrentalgorithmshavebeenselectedandtuned,inpartusingtests justlikethosedescribedhere,tobeasefficientasweknowhowtomakethem,whilestillofferingawiderangeof functionality.[6]ThemainreasonBoundedBufferfarespoorlyisthatputandtakeeachhavemultipleoperationsthat could encounter contentionͲacquire a semaphore, acquire a lock, release a semaphore. Other implementation approacheshavefewerpointsatwhichtheymightcontendwithanotherthread. [6]Youmightbeabletooutperformthemifyoubothareaconcurrencyexpertandcangiveupsomeoftheprovidedfunctionality. Figure12.2showscomparativethroughputonadualhyperͲthreadedmachineforallthreeclasseswith256Ͳelement buffers, using a variant of TimedPutTakeTest. This test suggests that LinkedBlockingQueue scales better than ArrayBlockingQueue.Thismayseemoddatfirst:alinkedqueuemustallocatealinknodeobjectforeachinsertion, andhenceseemstobedoingmoreworkthanthearrayͲbasedqueue.However,eventhoughithasmoreallocationand GCoverhead,alinkedqueueallowsmoreconcurrentaccessbyputsandtakesthananarrayͲbasedqueuebecausethe best linked queue algorithms allow the head and tail to be updated independently. Because allocation is usually threadlocal, algorithms that can reduce contention by doing more allocation usually scale better. (This is another instanceinwhichintuitionbasedontraditionalperformancetuningrunscountertowhatisneededforscalability.) Figure12.2.ComparingBlockingQueueImplementations.  12.2.3.MeasuringResponsiveness So far we have focused on measuring throughput, which is usually the most important performance metric for concurrent programs. But sometimes it is more important to know how long an individual action might take to complete,andinthiscasewewanttomeasurethevarianceofservicetime.Sometimesitmakessensetoallowalonger averageservicetimeifitletsusobtainasmallervariance;predictabilityisavaluableperformancecharacteristictoo. Measuring variance allows us to estimate the answers to qualityͲofͲservice questions like "What percentage of operationswillsucceedinunder100milliseconds?"  164 JavaConcurrencyInPractice Histogramsoftaskcompletiontimesarenormallythebestwaytovisualizevarianceinservicetime.Variancesareonly slightlymoredifficulttomeasurethanaveragesͲyouneedtokeeptrackofperͲtaskcompletiontimesinadditionto aggregatecompletiontime.Sincetimergranularitycanbeafactorinmeasuringindividualtasktime(anindividualtask maytakelessthanorclosetothesmallest"timertick",whichwoulddistortmeasurementsoftaskduration),toavoid measurementartifactswecanmeasuretheruntimeofsmallbatchesofputandtakeoperationsinstead. Figure12.3showstheperͲtaskcompletiontimesofavariantofTimedPutTakeTestusingabuffersizeof1000inwhich each of 256 concurrent tasks iterates only 1000 items for nonͲfair (shaded bars) and fair semaphores (open bars). (Section13.3explainsfairversusnonͲfairqueuingforlocksandsemaphores.)CompletiontimesfornonͲfairsemaphores range from 104 to 8,714 ms, a factor of over eighty. It is possible to reduce this range by forcing more fairness in concurrencycontrol;thisiseasytodoinBoundedBufferbyinitializingthesemaphorestofairmode.AsFigure12.3 shows,thissucceedsingreatlyreducingthevariance(nowrangingonlyfrom38,194to38,207ms),butunfortunately alsogreatlyreducesthethroughput.(AlongerͲrunningtestwithmoretypicalkindsoftaskswouldprobablyshowan evenlargerthroughputreduction.) Figure12.3.CompletionTimeHistogramforTimedPutTakeTestwithDefault(NonǦfair)andFairSemaphores.  Wesawbeforethatverysmallbuffersizescauseheavycontextswitchingandpoorthroughputeveninnonfairmode, becausenearlyeveryoperationinvolvesacontextswitch.Asanindicationthatthecostoffairnessresultsprimarilyfrom blocking threads, we can rerun this test with a buffer size of one and see that nonfair semaphores now perform comparablytofairsemaphores.Figure12.4showsthatfairnessdoesn'tmaketheaveragemuchworseorthevariance muchbetterinthiscase. Figure12.4.CompletionTimeHistogramforTimedPutTakeTestwithSingleǦitemBuffers.  So,unlessthreadsarecontinuallyblockinganywaybecauseoftightsynchronizationrequirements,nonfairsemaphores providemuchbetterthroughputandfairsemaphoresprovideslowervariance.Becausetheresultsaresodramatically different,Semaphoreforcesitsclientstodecidewhichofthetwofactorstooptimizefor.  1656BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms 12.3.AvoidingPerformanceTestingPitfalls Intheory,developingperformancetestsiseasyfindatypicalusagescenario,writeaprogramthatexecutesthatscenario manytimes,andtimeit.Inpractice,youhavetowatchoutforanumberofcodingpitfallsthatpreventperformance testsfromyieldingmeaningfulresults. 12.3.1.GarbageCollection Thetimingofgarbagecollectionisunpredictable,sothereisalwaysthepossibilitythatthegarbagecollectorwillrun duringameasuredtestrun.IfatestprogramperformsNiterationsandtriggersnogarbagecollectionbutiterationN+ 1wouldtriggeragarbagecollection,asmallvariationinthesizeoftheruncouldhaveabig(butspurious)effectonthe measuredtimeperiteration. There are two strategies for preventing garbage collection from biasing your results. One is to ensure that garbage collectiondoesnotrunatallduringyourtest(youcaninvoketheJVMwith-verbose:gctofindout);alternatively,you canmakesurethatthegarbagecollectorrunsanumberoftimesduringyourrunsothatthetestprogramadequately reflectsthecostofongoingallocationandgarbagecollection.Thelatterstrategyisoftenbetteritrequiresalongertest andismorelikelytoreflectrealͲworldperformance. MostproducerͲconsumerapplicationsinvolveafairamountofallocationandgarbagecollectionproducersallocatenew objectsthatareusedanddiscardedbyconsumers.Runningtheboundedbuffertestforlongenoughtoincurmultiple garbagecollectionsyieldsmoreaccurateresults. 12.3.2.DynamicCompilation WritingandinterpretingperformancebenchmarksfordynamicallycompiledlanguageslikeJavaisfarmoredifficultthan for statically compiled languages like C or C++. The HotSpot JVM (and other modern JVMs) uses a combination of bytecodeinterpretationanddynamiccompilation.Whenaclassisfirstloaded,theJVMexecutesitbyinterpretingthe bytecode.Atsomepoint,ifamethodisrunoftenenough,thedynamiccompilerkicksinandconvertsittomachine code;whencompilationcompletes,itswitchesfrominterpretationtodirectexecution. Thetimingofcompilationisunpredictable.Yourtimingtestsshouldrunonlyafterallcodehasbeencompiled;thereis no value in measuring the speed of the interpreted code since most programs run long enough that all frequently executedcodepathsarecompiled.Allowingthecompilertorunduringameasuredtestruncanbiastestresultsintwo ways:compilationconsumesCPUresources,andmeasuringtheruntimeofacombinationofinterpretedandcompiled codeisnotameaningfulperformancemetric.Figure12.5showshowthiscanbiasyourresults.Thethreetimelines representtheexecutionofthesamenumberofiterations:timelineArepresentsallinterpretedexecution,Brepresents compilationhalfwaythroughtherun,andCrepresentscompilationearlyintherun.Thepointatwhichcompilationruns seriouslyaffectsthemeasuredperͲoperationruntime.[7] [7]TheJVMmaychoosetoperformcompilationintheapplicationthreadorinthebackgroundthread;eachcanbiastimingresultsindifferent ways. Figure12.5.ResultsBiasedbyDynamicCompilation.  Codemayalsobedecompiled(revertingtointerpretedexecution)andrecompiledforvariousreasons,suchasloadinga classthatinvalidatesassumptionsmadebypriorcompilations,orgatheringsufficientprofilingdatatodecidethata codepathshouldberecompiledwithdifferentoptimizations. One way to prevent compilation from biasing your results is to run your program for a long time (at least several minutes) so that compilation and interpreted execution represent a small fraction of the total run time. Another approachistouseanunmeasured"warmͲup"run,inwhichyourcodeisexecutedenoughtobefullycompiledwhen  166 JavaConcurrencyInPractice youactuallystarttiming.OnHotSpot,runningyourprogramwith-XX:+PrintCompilationprintsoutamessagewhen dynamiccompilationruns,soyoucanverifythatthisispriorto,ratherthanduring,measuredtestruns. RunningthesametestseveraltimesinthesameJVMinstancecanbeusedtovalidatethetestingmethodology.Thefirst groupofresultsshouldbediscardedaswarmͲup;seeinginconsistentresultsintheremaininggroupssuggeststhatthe testshouldbeexaminedfurthertodeterminewhythetimingresultsarenotrepeatable. TheJVMusesvariousbackgroundthreadsforhousekeepingtasks.Whenmeasuringmultipleunrelatedcomputationally intensiveactivitiesinasinglerun,itisagoodideatoplaceexplicitpausesbetweenthemeasuredtrialstogivetheJVM achancetocatchupwithbackgroundtaskswithminimalinterferencefrommeasuredtasks.(Whenmeasuringmultiple relatedactivities,however,suchasmultiplerunsofthesametest,excludingJVMbackgroundtasksinthiswaymaygive unrealisticallyoptimisticresults.) 12.3.3.UnrealisticSamplingofCodePaths Runtimecompilersuseprofilinginformationtohelpoptimizethecodebeingcompiled.TheJVMispermittedtouse informationspecifictotheexecutioninordertoproducebettercode,whichmeansthatcompilingmethodMinone programmaygeneratedifferentcodethancompilingMinanother.Insomecases,theJVMmaymakeoptimizations basedonassumptionsthatmayonlybetruetemporarily,andlaterbackthemoutbyinvalidatingthecompiledcodeif theybecomeuntrue.[8] [8]Forexample,theJVMcanusemonomorphiccalltransformationtoconvertavirtualmethodcalltoadirectmethodcallifnoclassescurrently loadedoverridethatmethod,butitinvalidatesthecompiledcodeifaclassissubsequentlyloadedthatoverridesthemethod. Asaresult,itisimportantthatyourtestprogramsnotonlyadequatelyapproximatetheusagepatternsofatypical application,butalsoapproximatethesetofcodepathsusedbysuchanapplication.Otherwise,adynamiccompiler couldmakespecialoptimizationstoapurelysingleͲthreadedtestprogramthatcouldnotbeappliedinrealapplications containingatleastoccasionalparallelism.Therefore,testsofmultithreadedperformanceshouldnormallybemixedwith testsofsingleͲthreadedperformance,evenifyouwanttomeasureonlysingleͲthreadedperformance.(Thisissuedoes notariseinTimedPutTakeTestbecauseeventhesmallesttestcaseusestwothreads.) 12.3.4.UnrealisticDegreesofContention Concurrentapplicationstendtointerleavetwoverydifferentsortsofwork:accessingshareddata,suchasfetchingthe nexttaskfromasharedworkqueue,andthreadͲlocalcomputation(executingthetask,assumingthetaskitselfdoesnot accessshareddata).Dependingontherelativeproportionsofthetwotypesofwork,theapplicationwillexperience differentlevelsofcontentionandexhibitdifferentperformanceandscalingbehaviors. IfNthreadsarefetchingtasksfromasharedworkqueueandexecutingthem,andthetasksarecomputeͲintensiveand longͲrunning(anddonotaccessshareddataverymuch),therewillbealmostnocontention;throughputisdominated bytheavailabilityofCPUresources.Ontheotherhand,ifthetasksareveryshortͲlived,therewillbealotofcontention fortheworkqueueandthroughputisdominatedbythecostofsynchronization. Toobtainrealisticresults,concurrentperformancetestsshouldtrytoapproximatethethreadͲlocalcomputationdone byatypicalapplicationinadditiontotheconcurrentcoordinationunderstudy.Ifthetheworkdoneforeachtaskinan application is significantly different in nature or scope from the test program, it is easy to arrive at unwarranted conclusionsaboutwheretheperformancebottleneckslie.WesawinSection11.5that,forlockͲbasedclassessuchas thesynchronized Mapimplementations,whetheraccesstothelockismostlycontendedormostlyuncontendedcan have a dramatic effect on throughput. The tests in that section do nothing but pound on the Map; even with two threads,allattemptstoaccesstheMaparecontended.However,ifanapplicationdidasignificantamountofthreadͲ localcomputationforeachtimeitaccessestheshareddatastructure,thecontentionlevelmightbelowenoughtooffer goodperformance. Inthisregard,TimedPutTakeTestmaybeapoormodelforsomeapplications.Sincetheworkerthreadsdonotdovery much,throughputisdominatedbycoordinationoverhead,andthisisnotnecessarilythecaseinallapplicationsthat exchangedatabetweenproducersandconsumersviaboundedbuffers. 12.3.5.DeadCodeElimination Oneofthechallengesofwritinggoodbenchmarks(inanylanguage)isthatoptimizingcompilersareadeptatspotting andeliminatingdeadcodeͲcodethathasnoeffectontheoutcome.Sincebenchmarksoftendon'tcomputeanything, theyareaneasytargetfortheoptimizer.Mostofthetime,itisagoodthingwhentheoptimizerprunesdeadcodefrom aprogram,butforabenchmarkthisisabigproblembecausethenyouaremeasuringlessexecutionthanyouthink.If you'relucky,theoptimizerwillpruneawayyourentireprogram,andthenitwillbeobviousthatyourdataisbogus.If  1676BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms you'reunlucky,deadͲcodeeliminationwilljustspeedupyourprogrambysomefactorthatcouldbeexplainedbyother means. DeadͲcodeeliminationisaprobleminbenchmarkingstaticallycompiledlanguagestoo,butdetectingthatthecompiler haseliminatedagoodchunkofyourbenchmarkisaloteasierbecauseyoucanlookatthemachinecodeandseethata partofyourprogramismissing.Withdynamicallycompiledlanguages,thatinformationisnoteasilyaccessible. ManymicroͲbenchmarksperformmuch"better"whenrunwithHotSpot's-servercompilerthanwith-client,notjust becausetheservercompilercanproducemoreefficientcode,butalsobecauseitismoreadeptatoptimizingdead code.Unfortunately,thedeadͲcodeeliminationthatmadesuchshortworkofyourbenchmarkwon'tdoquiteaswell withcodethatactuallydoessomething.Butyoushouldstillprefer-serverto-clientforbothproductionandtesting onmultiprocessorsystemsͲyoujusthavetowriteyourtestssothattheyarenotsusceptibletodeadͲcodeelimination. Writingeffectiveperformancetestsrequirestrickingtheoptimizerintonotoptimizingawayyourbenchmarkasdead code. This requires every computed result to be used somehow by your programͲin a way that does not require synchronizationorsubstantialcomputation. InPutTakeTest,wecomputethechecksumofelementsaddedtoandremovedfromthequeueandcombinethese checksumsacrossallthethreads,butthiscouldstillbeoptimizedawayifwedonotactuallyusethechecksumvalue. Wehappentoneedittoverifythecorrectnessofthealgorithm,butyoucanensurethatavalueisusedbyprintingit out. However, you should avoid doing I/O while the test is actually running, so as not to distort the run time measurement. A cheap trick for preventing a calculation from being optimized away without introducing too much overhead is to computethehashCodeofthefieldofsomederivedobject,compareittoanarbitraryvaluesuchasthecurrentvalueof System. nanoTime,andprintauselessandignorablemessageiftheyhappentomatch: if (foo.x.hashCode() == System.nanoTime()) System.out.print(" "); Thecomparisonwillrarelysucceed,andifitdoes,itsonlyeffectwillbetoinsertaharmlessspacecharacterintothe output. (The print method buffers output until println is called, so in the rare case that hashCode and System.nanoTimeareequalnoI/Oisactuallyperformed.) Notonlyshouldeverycomputedresultbeused,butresultsshouldalsobeunguessable.Otherwise,asmartdynamic optimizingcompilerisallowedtoreplaceactionswithpreͲcomputedresults.Weaddressedthisintheconstructionof PutTakeTest,butanytestprogramwhoseinputisstaticdataisvulnerabletothisoptimization. 12.4.ComplementaryTestingApproaches Whilewe'dliketobelievethataneffectivetestingprogramshould"findallthebugs",thisisanunrealisticgoal.NASA devotesmoreofitsengineeringresourcestotesting(itisestimatedtheyemploy20testersforeachdeveloper)thanany commercialentitycouldaffordtoandthecodeproducedisstillnotfreeofdefects.Incomplexprograms,noamountof testingcanfindallcodingerrors. Thegoaloftestingisnotsomuchtofinderrorsasitistoincreaseconfidencethatthecodeworksasexpected.Sinceit isunrealistictoassumeyoucanfindallthebugs,thegoalofaqualityassurance(QA)planshouldbetoachievethe greatestpossibleconfidencegiventhetestingresourcesavailable.Morethingscangowronginaconcurrentprogram thaninasequentialone,andthereforemoretestingisrequiredtoachievethesamelevelofconfidence.Sofarwe've focusedprimarilyontechniquesforconstructingeffectiveunitandperformancetests.Testingiscriticallyimportantfor building confidence that concurrent classes behave correctly, but should be only one of the QA metholologies you employ. DifferentQAmethodologiesaremoreeffectiveatfindingsometypesofdefectsandlesseffectiveatfindingothers.By employing complementary testing methodologies such as code review and static analysis, you can achieve greater confidencethanyoucouldwithanysingleapproach. 12.4.1.CodeReview Aseffectiveandimportantasunitandstresstestsareforfindingconcurrencybugs,theyarenosubstituteforrigorous codereviewbymultiplepeople.(Ontheotherhand,codereviewisnosubstitutefortestingeither.)Youcanandshould designteststomaximizetheirchancesofdiscoveringsafetyerrors,andyoushouldrunthemfrequently,butyoushould notneglecttohaveconcurrentcodereviewedcarefullybysomeonebesidesitsauthor.Evenconcurrencyexpertsmake  168 JavaConcurrencyInPractice mistakes; taking the time to have someone else review the code is almost always worthwhile. Expert concurrent programmers are better at finding subtle races than are most test programs. (Also, platform issues such as JVM implementation details or processor memory models can prevent bugs from showing up on particular hardware or softwareconfigurations.) Codereviewalsohasotherbenefits; notonlycanitfinderrors,butitoftenimprovesthe qualityofcommentsdescribingtheimplementationdetails,thusreducingfuturemaintenancecostandrisk. 12.4.2.StaticAnalysisTools As of this writing, static analysis tools are rapidly emerging as an effective complement to formal testing and code review.Staticcodeanalysisistheprocessofanalyzingcodewithoutexecutingit,andcodeauditingtoolscananalyze classestolookforinstancesofcommonbugpatterns.StaticanalysistoolssuchastheopenͲsourceFindBugs[9]contain bugͲpatterndetectorsformanycommoncodingerrors,manyofwhichcaneasilybemissedbytestingorcodereview. [9]http://findbugs.sourceforge.net Staticanalysistoolsproducealistofwarningsthatmustbeexaminedbyhandtodeterminewhethertheyrepresent actualerrors.Historically,toolslikelintproducedsomanyfalsewarningsastoscaredevelopersaway,buttoolslike FindBugs have been tuned to produce many fewer false alarms. Static analysis tools are still somewhat primitive (especially in their integration with development tools and lifecycle), but they are already effective enough to be a valuableadditiontothetestingprocess. Asofthiswriting,FindBugsincludesdetectorsforthefollowingconcurrencyͲrelatedbugpatterns,andmorearebeing addedallthetime: Inconsistentsynchronization.Manyobjectsfollowthesynchronizationpolicyofguardingallvariableswiththeobject's intrinsic lock. If a field is accessed frequently but not always with the this lock held, this may indicate that the synchronizationpolicyisnotbeingconsistentlyfollowed. Analysis tools must guess at the synchronization policy because Java classes do not have formal concurrency specifications. In the future, if annotations such as @GuardedBy are standardized, auditing tools could interpret annotationsratherthanhavingtoguessattherelationshipbetweenvariablesandlocks,thusimprovingthequalityof analysis. InvokingThread.run.ThreadimplementsRunnableandthereforehasarunmethod.However,itisalmostalwaysa mistaketocallThread.rundirectly;usuallytheprogrammermeanttocallThread.start. Unreleasedlock.Unlikeintrinsiclocks,explicitlocks(seeChapter13)arenotautomaticallyreleasedwhencontrolexits thescopeinwhichtheywereacquired.Thestandardidiomistoreleasethelockfromafinallyblock;otherwisethe lockcanremainunreleasedintheeventofanException. Emptysynchronizedblock.WhileemptysynchronizedblocksdohavesemanticsundertheJavaMemoryModel,they arefrequentlyusedincorrectly,andthereareusuallybettersolutionstowhateverproblemthedeveloperwastryingto solve. DoubleͲchecked locking. DoubleͲchecked locking is a broken idiom for reducing synchronization overhead in lazy initialization(seeSection16.2.4)thatinvolvesreadingasharedmutablefieldwithoutappropriatesynchronization. Startingathreadfromaconstructor.Startingathreadfromaconstructorintroducestheriskofsubclassingproblems, andcanallowthethisreferencetoescapetheconstructor. Notificationerrors.ThenotifyandnotifyAllmethodsindicatethatanobject'sstatemayhavechangedinawaythat wouldunblockthreadsthatarewaitingontheassociatedconditionqueue.Thesemethodsshouldbecalledonlywhen thestateassociatedwiththeconditionqueuehaschanged.AsynchronizedblockthatcallsnotifyornotifyAllbut doesnotmodifyanystateislikelytobeanerror.(SeeChapter14.) Conditionwaiterrors.Whenwaitingonaconditionqueue,Object.waitorCondition. awaitshouldbecalledina loop, with the appropriate lock held, after testing some state predicate (see Chapter 14). Calling Object.wait or Condition.awaitwithoutthelockheld,notinaloop,orwithouttestingsomestatepredicateisalmostcertainlyan error. MisuseofLockandCondition.UsingaLockasthelockargumentforasynchronizedblockislikelytobeatypo,asis callingCondition.waitinsteadofawait(thoughthelatterwouldlikelybecaughtintesting,sinceitwouldthrowan IllegalMonitorStateExceptionthefirsttimeitwascalled).  1696BPartIII:Liveness,Performance,andTesting Ͳ 24BChapter12.TestingConcurrentPrograms Sleepingorwaitingwhileholdingalock.CallingThread.sleepwithalockheldcanpreventotherthreadsfrommaking progressforalongtimeandisthereforeapotentiallyseriouslivenesshazard.CallingObject.waitorCondition.await withtwolocksheldposesasimilarhazard. Spinloops.Codethatdoesnothingbutspin(busywait)checkingafieldforanexpectedvaluecanwasteCPUtimeand,if thefieldisnotvolatile,isnotguaranteedtoterminate.Latchesorconditionwaitsareoftenabettertechniquewhen waitingforastatetransitiontooccur. 12.4.3.AspectǦorientedTestingTechniques As of this writing, aspectͲoriented programming (AOP) techniques have only limited applicability to concurrency, becausemostpopularAOPtoolsdonotyetsupportpointcutsatsynchronizationpoints.However,AOPcanbeapplied toassertinvariantsorsomeaspectsofcompliancewithsynchronizationpolicies.Forexample,(Laddad,2003)provides anexampleofusinganaspecttowrapallcallstononͲthreadͲsafeSwingmethodswiththeassertionthatthecallis occurringintheeventthread.Asitrequiresnocodechanges,thistechniqueiseasytoapplyandcandisclosesubtle publicationandthreadͲconfinementerrors. 12.4.4.ProfilersandMonitoringTools Mostcommercialprofilingtoolshavesomesupportforthreads.Theyvaryinfeaturesetandeffectiveness,butcan oftenprovideinsightintowhatyourprogramisdoing(althoughprofilingtoolsareusuallyintrusiveandcansubstantially affectprogramtimingandbehavior).Mostofferadisplayshowingatimelineforeachthreadwithdifferentcolorsfor thevariousthreadstates(runnable,blockedwaitingforalock,blockedwaitingforI/O,etc.).Suchadisplaycanshow howeffectivelyyourprogramisutilizingtheavailableCPUresources,andifitisdoingbadly,wheretolookforthe cause. (Many profilers also claim features for identifying which locks are causing contention, but in practice these featuresareoftenablunterinstrumentthanisdesiredforanalyzingaprogram'slockingbehavior.) ThebuiltͲinJMXagentalsoofferssomelimitedfeaturesformonitoringthreadbehavior.TheThreadInfoclassincludes thethread'scurrentstateand,ifthethreadisblocked,thelockorconditionqueueonwhichitisblocked.Ifthe"thread contentionmonitoring"featureisenabled(itisdisabledbydefaultbecauseofitsperformanceimpact),ThreadInfoalso includesthenumberoftimesthatthethreadhasblockedwaitingforalockornotification,andthecumulativeamount oftimeithasspentwaiting. Summary Testingconcurrentprogramsforcorrectnesscanbeextremelychallengingbecausemanyofthepossiblefailuremodes of concurrent programs are lowͲprobability events that are sensitive to timing, load, and other hardͲtoͲreproduce conditions.Further,thetestinginfrastructurecanintroduceadditionalsynchronizationortimingconstraintsthatcan mask concurrency problems in the code being tested. Testing concurrent programs for performance can be equally challenging; Java programs are more difficult to test than programs written in statically compiled languages like C, becausetimingmeasurementscanbeaffectedbydynamiccompilation,garbagecollection,andadaptiveoptimization. Tohavethebestchanceoffindinglatentbugsbeforetheyoccurinproduction,combinetraditionaltestingtechniques (being careful to avoid the pitfalls discussed here) with code reviews and automated analysis tools. Each of these techniquesfindsproblemsthattheothersarelikelytomiss.   170 JavaConcurrencyInPractice PartIV:AdvancedTopics  Chapter13.ExplicitLocks Chapter14.BuildingCustomSynchronizers Chapter15.AtomicVariablesandNonͲblockingSynchronization Chapter16.TheJavaMemoryModel AppendixA.AnnotationsforConcurrency Bibliography  1717BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks Chapter13ǦExplicitLocks BeforeJava5.0,theonlymechanismsforcoordinatingaccesstoshareddataweresynchronizedandvolatile.Java 5.0addsanotheroption:ReentrantLock.Contrarytowhatsomehavewritten,ReentrantLockisnotareplacementfor intrinsiclocking,butratheranalternativewithadvancedfeaturesforwhenintrinsiclockingprovestoolimited. 13.1.LockandReentrantLock TheLockinterface,showninListing13.1,definesanumberofabstractlockingoperations.Unlikeintrinsiclocking,Lock offersachoiceofunconditional,polled,timed,andinterruptiblelockacquisition,andalllockandunlockoperationsare explicit.LockimplementationsmustprovidethesamememoryͲvisibilitysemanticsasintrinsiclocks,butcandifferin their locking semantics, scheduling algorithms, ordering guarantees, and performance characteristics. (Lock.newConditioniscoveredinChapter14.) Listing13.1.LockInterface. public interface Lock { void lock(); void lockInterruptibly() throws InterruptedException; boolean tryLock(); boolean tryLock(long timeout, TimeUnit unit) throws InterruptedException; void unlock(); Condition newCondition(); } ReentrantLock implements Lock, providing the same mutual exclusion and memoryͲvisibility guarantees as synchronized.Acquiringa ReentrantLockhasthesamememorysemanticsasenteringa synchronizedblock,and releasing a ReentrantLock has the same memory semantics as exiting a synchronized block. (Memory visibility is coveredinSection3.1andinChapter16.)And,likesynchronized,ReentrantLockoffersreentrantlockingsemantics (seeSection2.3.2).ReentrantLocksupportsallofthelockͲacquisitionmodesdefinedbyLock,providingmoreflexibility fordealingwithlockunavailabilitythandoessynchronized. Whycreateanewlockingmechanismthatissosimilartointrinsiclocking?Intrinsiclockingworksfineinmostsituations buthassomefunctionallimitationsͲitisnotpossibletointerruptathreadwaitingtoacquirealock,ortoattemptto acquirealockwithoutbeingwillingtowaitforitforever.Intrinsiclocksalsomustbereleasedinthesameblockofcode inwhichtheyareacquired;thissimplifiescodingandinteractsnicelywithexceptionhandling,butmakesnonͲblockͲ structuredlockingdisciplinesimpossible.Noneofthesearereasonstoabandon synchronized,butinsomecasesa moreflexiblelockingmechanismoffersbetterlivenessorperformance. Listing13.2showsthecanonicalformforusingaLock.Thisidiomissomewhatmorecomplicatedthanusingintrinsic locks:thelockmustbereleasedinafinallyblock.Otherwise,thelockwouldneverbereleasediftheguardedcode weretothrowanexception.Whenusinglocking,youmustalsoconsiderwhathappensifanexceptionisthrownoutof thetryblock;ifitispossiblefortheobjecttobeleftinaninconsistentstate,additionalTRy-catchorTRy-finally blocksmaybeneeded.(Youshouldalwaysconsidertheeffectofexceptionswhenusinganyformoflocking,including intrinsiclocking.) FailingtousefinallytoreleaseaLockisatickingtimebomb.Whenitgoesoff,youwillhaveahardtimetracking downitsoriginastherewillbenorecordofwhereorwhentheLockshouldhavebeenreleased.Thisisonereasonnot to use ReentrantLock as a blanket substitute for synchronized: it is more "dangerous" because it doesn't automaticallycleanupthelockwhencontrolleavestheguardedblock.Whilerememberingtoreleasethelockfroma finallyblockisnotallthatdifficult,itisalsonotimpossibletoforget.[1] [1]FindBugshasan"unreleasedlock"detectoridentifyingwhenaLockisnotreleasedinallcodepathsoutoftheblockinwhichitwasacquired. Listing13.2.GuardingObjectStateUsingReentrantLock. Lock lock = new ReentrantLock(); ... lock.lock(); try { // update object state // catch exceptions and restore invariants if necessary } finally { lock.unlock(); }   172 JavaConcurrencyInPractice 13.1.1.PolledandTimedLockAcquisition The timed and polled lockͲacquisition modes provided by TryLock allow more sophisticated error recovery than unconditionalacquisition.Withintrinsiclocks,adeadlockisfatalͲtheonlywaytorecoveristorestarttheapplication, andtheonlydefenseistoconstructyourprogramsothatinconsistentlockorderingisimpossible.Timedandpolled lockingofferanotheroption:probabilisticdeadlockavoidance. Usingtimedorpolledlockacquisition(TryLock)letsyouregaincontrolifyoucannotacquirealltherequiredlocks, releasetheonesyoudidacquire,andtryagain(oratleastlogthefailureanddosomethingelse).Listing13.3showsan alternatewayofaddressingthedynamicorderingdeadlockfromSection10.1.2:use TRyLocktoattempttoacquire bothlocks,butbackoffandretryiftheycannotbothbeacquired.Thesleeptimehasafixedcomponentandarandom component to reduce the likelihood of livelock. If the locks cannot be acquired within the specified time, transferMoneyreturnsafailurestatussothattheoperationcanfailgracefully.(See[CPJ2.5.1.2]and[CPJ2.5.1.3]for moreexamplesofusingpolledlocksfordeadlockavoidance.) Timedlocksarealsousefulinimplementingactivitiesthatmanageatimebudget(seeSection6.3.7).Whenanactivity withatimebudgetcallsablockingmethod,itcansupplyatimeoutcorrespondingtotheremainingtimeinthebudget. Thisletsactivitiesterminateearlyiftheycannotdeliveraresultwithinthedesiredtime.Withintrinsiclocks,thereisno waytocancelalockacquisitiononceitisstarted,sointrinsiclocksputtheabilitytoimplementtimeͲbudgetedactivities atrisk. ThetravelportalexampleinListing6.17onpage134createsaseparatetaskforeachcarͲrentalcompanyfromwhichit wassolicitingbids.SolicitingabidprobablyinvolvessomesortofnetworkͲbasedrequestmechanism,suchasaweb service request. But soliciting a bid might also require exclusive access to a scarce resource, such as a direct communicationslinetothecompany. WesawonewaytoensureserializedaccesstoaresourceinSection9.5:asingleͲthreadedexecutor.Anotherapproach istouseanexclusivelocktoguardaccesstotheresource.ThecodeinListing13.4triestosendamessageonashared communicationslineguardedbyaLock,butfailsgracefullyifitcannotdosowithinitstimebudget.ThetimedTRyLock makesitpracticaltoincorporateexclusivelockingintosuchatimeͲlimitedactivity. 13.1.2.InterruptibleLockAcquisition Just as timed lock acquisition allows exclusive locking to be used within timeͲlimited activities, interruptible lock acquisitionallowslockingtobeusedwithincancellableactivities.Section7.1.6identifiedseveralmechanisms,suchas acquiring an intrinsic lock, that are not responsive to interruption. These nonͲinterruptible blocking mechanisms complicatetheimplementationofcancellabletasks.ThelockInterruptiblymethodallowsyoutotrytoacquirealock while remaining responsive to interruption, and its inclusion in Lock avoids creating another category of nonͲ interruptibleblockingmechanisms.  1737BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks Listing13.3.AvoidingLockǦorderingDeadlockUsingtrylock. public boolean transferMoney(Account fromAcct, Account toAcct, DollarAmount amount, long timeout, TimeUnit unit) throws InsufficientFundsException, InterruptedException { long fixedDelay = getFixedDelayComponentNanos(timeout, unit); long randMod = getRandomDelayModulusNanos(timeout, unit); long stopTime = System.nanoTime() + unit.toNanos(timeout); while (true) { if (fromAcct.lock.tryLock()) { try { if (toAcct.lock.tryLock()) { try { if (fromAcct.getBalance().compareTo(amount) < 0) throw new InsufficientFundsException(); else { fromAcct.debit(amount); toAcct.credit(amount); return true; } } finally { toAcct.lock.unlock(); } } } finally { fromAcct.lock.unlock(); } } if (System.nanoTime() < stopTime) return false; NANOSECONDS.sleep(fixedDelay + rnd.nextLong() % randMod); } } Listing13.4.LockingwithaTimeBudget. public boolean trySendOnSharedLine(String message, long timeout, TimeUnit unit) throws InterruptedException { long nanosToLock = unit.toNanos(timeout) - estimatedNanosToSend(message); if (!lock.tryLock(nanosToLock, NANOSECONDS)) return false; try { return sendOnSharedLine(message); } finally { lock.unlock(); } } Thecanonicalstructureofinterruptiblelockacquisitionisslightlymorecomplicatedthannormallockacquisition,astwo TRy blocks are needed. (If the interruptible lock acquisition can throw InterruptedException, the standard try- finallylockingidiomworks.)Listing13.5useslockInterruptiblytoimplementsendOnSharedLinefromListing13.4 sothatwecancallitfromacancellabletask.ThetimedTRyLockisalsoresponsivetointerruptionandsocanbeused whenyouneedbothtimedandinterruptiblelockacquisition. Listing13.5.InterruptibleLockAcquisition. public boolean sendOnSharedLine(String message) throws InterruptedException { lock.lockInterruptibly(); try { return cancellableSendOnSharedLine(message); } finally { lock.unlock(); } } private boolean cancellableSendOnSharedLine(String message) throws InterruptedException { ... } 13.1.3.NonǦblockǦstructuredLocking Withintrinsiclocks,acquireͲreleasepairsareblockͲstructuredalockisalwaysreleasedinthesamebasicblockinwhich it was acquired, regardless of how control exits the block. Automatic lock release simplifies analysis and prevents potentialcodingerrors,butsometimesamoreflexiblelockingdisciplineisneeded.  174 JavaConcurrencyInPractice InChapter11,wesawhowreducinglockgranularitycanenhancescalability.Lockstripingallowsdifferenthashchainsin ahashͲbasedcollectiontousedifferentlocks.Wecanapplyasimilarprincipletoreducelockinggranularityinalinked list by using a separate lock for each link node, allowing different threads to operate independently on different portions of the list. The lock for a given node guards the link pointers and the data stored in that node, so when traversingormodifyingthelistwemustholdthelockononenodeuntilweacquirethelockonthenextnode;onlythen canwereleasethelockonthefirstnode.Anexampleofthistechnique,calledhandͲoverͲhandlockingorlockcoupling, appearsin[CPJ2.5.1.4]. 13.2.PerformanceConsiderations When ReentrantLockwasaddedinJava5.0,itofferedfarbettercontendedperformancethanintrinsiclocking.For synchronizationprimitives,contendedperformanceisthekeytoscalability:ifmoreresourcesareexpendedonlock managementandscheduling,fewerareavailablefortheapplication.Abetterlockimplementationmakesfewersystem calls, forces fewer context switches, and initiates less memoryͲsynchronization traffic on the shared memory bus, operationsthataretimeͲconsuminganddivertcomputingresourcesfromtheprogram. Java6usesanimprovedalgorithmformanagingintrinsiclocks,similartothatusedbyReentrantLock,thatclosesthe scalabilitygapconsiderably.Figure13.1showstheperformancedifferencebetweenintrinsiclocksandReentrantLock onJava5.0andonaprereleasebuildofJava6onafourͲwayOpteronsystemrunningSolaris.Thecurvesrepresentthe "speedup" of ReentrantLock over intrinsic locking on a single JVM version. On Java 5.0, ReentrantLock offers considerablybetterthroughput,butonJava6,thetwoarequiteclose.[2]Thetestprogramisthesameoneusedin Section11.5,thistimecomparingthethroughputofaHashMapguardedbyanintrinsiclockandbyaReentrantLock. [2]Thoughthisparticulargraphdoesn'tshowit,thescalabilitydifferencebetweenJava5.0andJava6reallydoescomefromimprovementin intrinsiclocking,ratherthanfromregressioninReentrant-Lock. Figure13.1.IntrinsicLockingVersusReentrantLockPerformanceonJava5.0andJava6.   OnJava5.0,theperformanceofintrinsiclockingdropsdramaticallyingoingfromonethread(nocontention)tomore thanonethread;theperformanceofReentrantLockdropsfarless,showingitsbetterscalability.ButonJava6,itisa differentstoryintrinsiclocksnolongerfallapartundercontention,andthetwoscalefairlysimilarly. GraphslikeFigure13.1remindusthatstatementsoftheform"XisfasterthanY"areatbestshortͲlived.Performance andscalabilityaresensitivetoplatformfactorssuchasCPU,processorcount,cachesize,andJVMcharacteristics,allof whichcanchangeovertime.[3] [3]Whenwestartedthisbook,ReentrantLockseemedthelastwordinlockscalability.Lessthanayearlater,intrinsiclockinggivesitagood runforitsmoney.Performanceisnotjustamovingtarget,itcanbeafastͲmovingtarget. Performanceisamovingtarget;yesterday'sbenchmarkshowingthatXisfasterthanYmayalreadybeoutofdate today.  1757BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks 13.3.Fairness TheReentrantLockconstructoroffersachoiceoftwofairnessoptions:createanonfairlock(thedefault)orafairlock. Threadsacquireafairlockintheorderinwhichtheyrequestedit,whereasanonfairlockpermitsbarging:threads requesting a lock can jump ahead of the queue of waiting threads if the lock happens to be available when it is requested.(Semaphorealsooffersthechoiceoffairornonfairacquisitionordering.)NonfairReentrantLocksdonotgo outoftheirwaytopromotebargingtheysimplydon'tpreventathreadfrombargingifitshowsupattherighttime. Withafairlock,anewlyrequestingthreadisqueuedifthelockisheldbyanotherthreadorifthreadsarequeued waitingforthelock;withanonfairlock,thethreadisqueuedonlyifthelockiscurrentlyheld.[4] [4]ThepolledtryLockalwaysbarges,evenforfairlocks. Wouldn'twewantalllockstobefair?Afterall,fairnessisgoodandunfairnessisbad,right?(Justaskyourkids.)Whenit comes to locking, though, fairness has a significant performance cost because of the overhead of suspending and resumingthreads.Inpractice,astatisticalfairnessguaranteeͲpromisingthatablockedthreadwilleventuallyacquire thelockͲisoftengoodenough,andisfarlessexpensivetodeliver.Somealgorithmsrelyonfairqueuingtoensuretheir correctness,buttheseareunusual.Inmostcases,theperformancebenefitsofnonͲfairlocksoutweighthebenefitsof fairqueuing. Figure13.2showsanotherrunoftheMapperformancetest,thistimecomparingHashMapwrappedwithfairandnonͲ fair ReentrantLocksonafourͲwayOpteronsystemrunningSolaris,plottedonalogscale.[5]Thefairnesspenaltyis nearlytwoordersofmagnitude.Don'tpayforfairnessifyoudon'tneedit. [5]ThegraphforConcurrentHashMapisfairlywigglyintheregionbetweenfourandeightthreads.Thesevariationsalmostcertainlycome frommeasurementnoise,whichcouldbeintroducedbycoincidentalinteractionswiththehashcodesoftheelements,threadscheduling,map resizing,garbagecollectionorothermemoryͲsystemeffects,orbytheOSdecidingtorunsomeperiodichousekeepingtaskaroundthetimethat testcaseran.Therealityisthatthereareallsortsofvariationsinperformanceteststhatusuallyaren'tworthbotheringtocontrol.Wemadeno attempttocleanupourgraphsartificially,becauserealͲworldperformancemeasurementsarealsofullofnoise. Figure13.2.FairVersusNonǦfairLockPerformance.   One reason barging locks perform so much better than fair locks under heavy contention is that there can be a significantdelaybetweenwhenasuspendedthreadisresumedandwhenitactuallyruns.Let'ssaythreadAholdsalock andthreadBasksforthatlock.Sincethelockisbusy,Bissuspended.WhenAreleasesthelock,Bisresumedsoitcan tryagain.Inthemeantime,though,ifthreadCrequeststhelock,thereisagoodchancethatCcanacquirethelock,use it,andreleaseitbeforeBevenfinisheswakingup.Inthiscase,everyonewins:Bgetsthelocknolaterthanitotherwise wouldhave,Cgetsitmuchearlier,andthroughputisimproved. Fairlockstendtoworkbestwhentheyareheldforarelativelylongtimeorwhenthemeantimebetweenlockrequests isrelativelylong.Inthesecases,theconditionunderwhichbargingprovidesathroughputadvantageͲwhenthelockis unheldbutathreadiscurrentlywakinguptoclaimitͲislesslikelytohold. LikethedefaultReentrantLock,intrinsiclockingoffersnodeterministicfairnessguarantees,butthestatisticalfairness guaranteesofmostlockingimplementationsaregoodenoughforalmostallsituations.Thelanguagespecificationdoes notrequiretheJVMtoimplementintrinsiclocksfairly,andnoproductionJVMsdo.ReentrantLockdoesnotdepress lockfairnesstonewlowsitonlymakesexplicitsomethingthatwaspresentallalong.  176 JavaConcurrencyInPractice 13.4.ChoosingBetweenSynchronizedandReentrantLock ReentrantLockprovidesthesamelockingandmemorysemanticsasintrinsiclocking,aswellasadditionalfeaturessuch astimedlockwaits,interruptiblelockwaits,fairness,andtheabilitytoimplementnonͲblockͲstructuredlocking.The performanceofReentrantLockappearstodominatethatofintrinsiclocking,winningslightlyonJava6anddramatically onJava5.0.SowhynotdeprecatesynchronizedandencourageallnewconcurrentcodetouseReentrantLock?Some authorshaveinfactsuggestedthis,treatingsynchronizedasa"legacy"construct.Butthisistakingagoodthingway toofar. Intrinsic locks still have significant advantages over explicit locks. The notation is familiar and compact, and many existingprogramsalreadyuseintrinsiclockingandmixingthetwocouldbeconfusinganderrorͲprone.Reentrant-Lock isdefinitelyamoredangeroustoolthansynchronization;ifyouforgettowraptheunlockcallinafinallyblock,your codewillprobablyappeartorunproperly,butyou'vecreatedatimebombthatmaywellhurtinnocentbystanders. SaveReentrantLockforsituationsinwhichyouneedsomethingReentrantLockprovidesthatintrinsiclockingdoesn't. ReentrantLockisanadvancedtoolforsituationswhereintrinsiclockingisnotpractical.Useitifyouneeditsadvanced features:timed,polled,orinterruptiblelockacquisition,fairqueuing,ornonͲblockͲstructuredlocking.Otherwise,prefer synchronized. UnderJava5.0,intrinsiclockinghasanotheradvantageover ReentrantLock:threaddumpsshowwhichcallframes acquiredwhichlocksandcandetectandidentifydeadlockedthreads.TheJVMknowsnothingaboutwhichthreadshold ReentrantLocksandthereforecannothelpindebuggingthreadingproblemsusing ReentrantLock.Thisdisparityis addressedinJava6byprovidingamanagementandmonitoringinterfacewithwhichlockscanregister,enablinglocking informationforReentrantLockstoappearinthreaddumpsandthroughothermanagementanddebugginginterfaces. Theavailabilityofthisinformationfordebuggingisasubstantial,ifmostlytemporary,advantagefor synchronized; lockinginformationinthreaddumpshassavedmanyprogrammersfromutterconsternation.ThenonͲblockͲstructured natureof ReentrantLockstillmeansthatlockacquisitionscannotbetiedtospecificstackframes,astheycanwith intrinsiclocks. Futureperformanceimprovementsarelikelytofavor synchronizedover ReentrantLock.Because synchronizedis builtintotheJVM,itcanperformoptimizationssuchaslockelisionforthreadͲconfinedlockobjectsandlockcoarsening toeliminatesynchronizationwithintrinsiclocks(seeSection11.3.2);doingthiswithlibraryͲbasedlocksseemsfarless likely. Unless you are deploying on Java 5.0 for the foreseeable future and you have a demonstrated need for ReentrantLock's scalability benefits on that platform, it is not a good idea to choose ReentrantLock over synchronizedforperformancereasons. 13.5.ReadǦwriteLocks ReentrantLockimplementsastandardmutualͲexclusionlock:atmostonethreadatatimecanholdaReentrantLock. Butmutualexclusionisfrequentlyastrongerlockingdisciplinethanneededtopreservedataintegrity,andthuslimits concurrencymorethannecessary.Mutualexclusionisaconservativelockingstrategythatpreventswriter/writerand writer/readeroverlap,butalsopreventsreader/readeroverlap.Inmanycases,datastructuresare"readͲmostly"they aremutableandaresometimesmodified,butmostaccessesinvolveonlyreading.Inthesecases,itwouldbeniceto relaxthelockingrequirementstoallowmultiplereaderstoaccessthedatastructureatonce.Aslongaseachthreadis guaranteedanupͲtoͲdateviewofthedataandnootherthreadmodifiesthedatawhilethereadersareviewingit,there willbenoproblems.ThisiswhatreadͲwritelocksallow:aresourcecanbeaccessedbymultiplereadersorasingle writeratatime,butnotboth. ReadWriteLock, shown in Listing 13.6, exposes two Lock objectsone for reading and one for writing. To read data guardedbyaReadWriteLockyoumustfirstacquirethereadlock,andtomodifydataguardedbyaReadWriteLockyou mustfirstacquirethewritelock.Whiletheremayappeartobetwoseparatelocks,thereadlockandwritelockare simplydifferentviewsofanintegratedreadͲwritelockobject. Listing13.6.ReadWriteLockInterface. public interface ReadWriteLock { Lock readLock(); Lock writeLock(); }   1777BPartIV:AdvancedTopics Ͳ25BChapter13Ͳ ExplicitLocks ThelockingstrategyimplementedbyreadͲwritelocksallowsmultiplesimultaneousreadersbutonlyasinglewriter.Like Lock, ReadWriteLock admits multiple implementations that can vary in performance, scheduling guarantees, acquisitionpreference,fairness,orlockingsemantics. ReadͲwritelocksareaperformanceoptimizationdesignedtoallowgreaterconcurrencyincertainsituations.Inpractice, readͲwrite locks can improve performance for frequently accessed readͲmostly data structures on multiprocessor systems; under other conditions they perform slightly worse than exclusive locks due to their greater complexity. Whethertheyareanimprovementinanygivensituationisbestdeterminedviaprofiling;becauseReadWriteLockuses Lockforthereadandwriteportionsofthelock,itisrelativelyeasytoswapoutareadͲwritelockforanexclusiveoneif profilingdeterminesthatareadͲwritelockisnotawin. The interaction between the read and write locks allows for a number of possible implementations. Some of the implementationoptionsforaReadWriteLockare: Releasepreference.Whenawriterreleasesthewritelockandbothreadersandwritersarequeuedup,whoshouldbe givenpreferenceͲreaders,writers,orwhoeveraskedfirst? Readerbarging.Ifthelockisheldbyreadersbuttherearewaitingwriters,shouldnewlyarrivingreadersbegranted immediate access, or should they wait behind the writers? Allowing readers to barge ahead of writers enhances concurrencybutrunstheriskofstarvingwriters. Reentrancy.Arethereadandwritelocksreentrant? Downgrading.Ifathreadholdsthewritelock,canitacquirethereadlockwithoutreleasingthewritelock?Thiswould letawriter"downgrade"toareadlockwithoutlettingotherwritersmodifytheguardedresourceinthemeantime. Upgrading.Canareadlockbeupgradedtoawritelockinpreferencetootherwaitingreadersorwriters?MostreadͲ writelockimplementationsdonotsupportupgrading,becausewithoutanexplicitupgradeoperationitisdeadlockͲ prone.(Iftworeaderssimultaneouslyattempttoupgradetoawritelock,neitherwillreleasethereadlock.) ReentrantReadWriteLock provides reentrant locking semantics for both locks. Like ReentrantLock, a ReentrantReadWriteLockcanbeconstructedasnonͲfair(thedefault)orfair.Withafairlock,preferenceisgiventothe threadthathasbeenwaitingthelongest;ifthelockisheldbyreadersandathreadrequeststhewritelock,nomore readersareallowedtoacquirethereadlockuntilthewriterhasbeenservicedandreleasesthewritelock.WithanonͲ fairlock,theorderinwhichthreadsaregrantedaccessisunspecified.Downgradingfromwritertoreaderispermitted; upgradingfromreadertowriterisnot(attemptingtodosoresultsindeadlock). LikeReentrantLock,thewritelockinReentrantReadWriteLockhasauniqueownerandcanbereleasedonlybythe threadthatacquiredit.InJava5.0,thereadlockbehavesmorelikeaSemaphorethanalock,maintainingonlythecount ofactivereaders,nottheiridentities.ThisbehaviorwaschangedinJava6tokeeptrackalsoofwhichthreadshavebeen grantedthereadlock.[6] [6]OnereasonforthischangeisthatunderJava5.0,thelockimplementationcannotdistinguishbetweenathreadrequestingthereadlockforthe firsttimeandareentrantlockrequest,whichwouldmakefairreadͲwritelocksdeadlockͲprone. ReadͲwritelockscanimproveconcurrencywhenlocksaretypicallyheldforamoderatelylongtimeandmostoperations donotmodifytheguardedresources.ReadWriteMapinListing13.7usesaReentrantReadWriteLocktowrapaMapso thatitcanbesharedsafelybymultiplereadersandstillpreventreaderͲwriterorwriterͲwriterconflicts.[7]Inreality, ConcurrentHashMap's performance is so good that you would probably use it rather than this approach if all you neededwasaconcurrenthashͲbasedmap,butthistechniquewouldbeusefulifyouwanttoprovidemoreconcurrent accesstoanalternateMapimplementationsuchasLinkedHashMap. [7]ReadWriteMapdoesnotimplementMapbecauseimplementingtheviewmethodssuchasentrySetandvalueswouldbedifficultand the"easy"methodsareusuallysufficient.  178 JavaConcurrencyInPractice Listing13.7.WrappingaMapwithaReadǦwriteLock. public class ReadWriteMap { private final Map map; private final ReadWriteLock lock = new ReentrantReadWriteLock(); private final Lock r = lock.readLock(); private final Lock w = lock.writeLock(); public ReadWriteMap(Map map) { this.map = map; } public V put(K key, V value) { w.lock(); try { return map.put(key, value); } finally { w.unlock(); } } // Do the same for remove(), putAll(), clear() public V get(Object key) { r.lock(); try { return map.get(key); } finally { r.unlock(); } } // Do the same for other read-only Map methods } Figure 13.3 shows a throughput comparison between an ArrayList wrapped with a ReentrantLock and with a ReadWriteLock on a fourͲway Opteron system running Solaris. The test program used here is similar to the Map performancetestwe'vebeenusingthroughoutthebookͲeachoperationrandomlyselectsavalueandsearchesforitin thecollection,andasmallpercentageofoperationsmodifythecontentsofthecollection. Figure13.3.ReadǦwriteLockPerformance.  Summary ExplicitLocksofferanextendedfeaturesetcomparedtointrinsiclocking,includinggreaterflexibilityindealingwith lock unavailability and greater control over queuing behavior. But ReentrantLock is not a blanket substitute for synchronized;useitonlywhenyouneedfeaturesthatsynchronizedlacks. ReadͲwritelocksallowmultiplereaderstoaccessaguardedobjectconcurrently,offeringthepotentialforimproved scalabilitywhenaccessingreadͲmostlydatastructures.  1797BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers Chapter14ǦBuildingCustomSynchronizers The class libraries include a number of stateͲdependent classesͲthose having operations with stateͲbased preconditionsͲsuchasFutureTask,Semaphore,andBlockingQueue.Forexample,youcannotremoveanitemfroman emptyqueueorretrievetheresultofataskthathasnotyetfinished;beforetheseoperationscanproceed,youmust waituntilthequeueentersthe"nonempty"stateorthetaskentersthe"completed"state. TheeasiestwaytoconstructastateͲdependentclassisusuallytobuildontopofanexistingstateͲdependentlibrary class;wedidthisinValueLatchonpage187,usingaCountDownLatchtoprovidetherequiredblockingbehavior.Butif thelibraryclassesdonotprovidethefunctionalityyouneed,youcanalsobuildyourownsynchronizersusingthelowͲ level mechanisms provided by the language and libraries, including intrinsic condition queues, explicit Condition objects,andtheAbstractQueuedSynchronizerframework.Thischapterexploresthevariousoptionsforimplementing statedependenceandtherulesforusingthestatedependencemechanismsprovidedbytheplatform. 14.1.ManagingStateDependence InasingleͲthreadedprogram,ifastateͲbasedprecondition(like"theconnectionpoolisnonempty")doesnotholdwhen amethodiscalled,itwillneverbecometrue.Therefore,classesinsequentialprogramscanbecodedtofailwhentheir preconditionsdonothold.Butinaconcurrentprogram,stateͲbasedconditionscanchangethroughtheactionsofother threads: a pool that was empty a few instructions ago can become nonempty because another thread returned an element. StateͲdependent methods on concurrent objects can sometimes get away with failing when their preconditionsarenotmet,butthereisoftenabetteralternative:waitforthepreconditiontobecometrue. StateͲdependentoperationsthatblockuntiltheoperationcanproceedaremoreconvenientandlesserrorͲpronethan thosethatsimplyfail.ThebuiltͲinconditionqueuemechanismenablesthreadstoblockuntilanobjecthasentereda statethatallowsprogressandtowakeblockedthreadswhentheymaybeabletomakefurtherprogress.Wecoverthe detailsofconditionqueuesinSection14.2,buttomotivatethevalueofanefficientconditionwaitmechanism,wefirst showhowstatedependencemightbe(painfully)tackledusingpollingandsleeping. AblockingstateͲdependentactiontakestheformshowninListing14.1.Thepatternoflockingissomewhatunusualin that the lock is released and reacquired in the middle of the operation. The state variables that make up the preconditionmustbeguardedbytheobject'slock,sothattheycanremainconstantwhilethepreconditionistested. Butifthepreconditiondoesnothold,thelockmustbereleasedsoanotherthreadcanmodifytheobjectstateotherwise thepreconditionwillneverbecometrue.Thelockmustthenbereacquiredbeforetestingthepreconditionagain. Listing14.1.StructureofBlockingStateǦdependentActions. void blockingAction() throws InterruptedException { acquire lock on object state while (precondition does not hold) { release lock wait until precondition might hold optionally fail if interrupted or timeout expires reacquire lock } perform action } BoundedbufferssuchasArrayBlockingQueuearecommonlyusedinproducerͲconsumerdesigns.Aboundedbuffer providesputandtakeoperations,eachofwhichhaspreconditions:youcannottakeanelementfromanemptybuffer, norputanelementintoafullbuffer.Statedependentoperationscandealwithpreconditionfailurebythrowingan exceptionorreturninganerrorstatus(makingitthecaller'sproblem),orbyblockinguntiltheobjecttransitionstothe rightstate. We're going to develop several implementations of a bounded buffer that take different approaches to handling preconditionfailure.EachextendsBaseBoundedBufferinListing14.2,whichimplementsaclassicarrayͲbasedcircular buffer where the buffer state variables (buf, head, tail, and count) are guarded by the buffer's intrinsic lock. It provides synchronized doPut and doTake methods that are used by subclasses to implement the put and take operations;theunderlyingstateishiddenfromthesubclasses. 14.1.1.Example:PropagatingPreconditionFailuretoCallers GrumpyBoundedBufferin Listing14.3isacrudefirstattemptat implementingaboundedbuffer.The putand take methodsaresynchronizedtoensureexclusiveaccesstothebufferstate,sincebothemploycheckͲthenͲactlogicin accessingthebuffer.  180 JavaConcurrencyInPractice Whilethisapproachiseasyenoughtoimplement,itisannoyingtouse.Exceptionsaresupposedtobeforexceptional conditions[EJItem39]."Bufferisfull"isnotanexceptionalconditionforaboundedbufferanymorethan"red"isan exceptionalconditionforatrafficsignal.Thesimplificationinimplementingthebuffer(forcingthecallertomanagethe statedependence)ismorethanmadeupforbythesubstantialcomplicationinusingit,sincenowthecallermustbe preparedtocatchexceptionsandpossiblyretryforeverybufferoperation.[1]AwellͲstructuredcalltotakeisshownin Listing14.4notverypretty,especiallyifputandtakearecalledthroughouttheprogram. [1]PushingthestatedependencebacktothecalleralsomakesitnearlyimpossibletodothingslikepreserveFIFOordering;byforcingthecallerto retry,youlosetheinformationofwhoarrivedfirst. Listing14.2.BaseClassforBoundedBufferImplementations. @ThreadSafe public abstract class BaseBoundedBuffer { @GuardedBy("this") private final V[] buf; @GuardedBy("this") private int tail; @GuardedBy("this") private int head; @GuardedBy("this") private int count; protected BaseBoundedBuffer(int capacity) { this.buf = (V[]) new Object[capacity]; } protected synchronized final void doPut(V v) { buf[tail] = v; if (++tail == buf.length) tail = 0; ++count; } protected synchronized final V doTake() { V v = buf[head]; buf[head] = null; if (++head == buf.length) head = 0; --count; return v; } public synchronized final boolean isFull() { return count == buf.length; } public synchronized final boolean isEmpty() { return count == 0; } } Listing14.3.BoundedBufferthatBalksWhenPreconditionsareNotMet. @ThreadSafe public class GrumpyBoundedBuffer extends BaseBoundedBuffer { public GrumpyBoundedBuffer(int size) { super(size); } public synchronized void put(V v) throws BufferFullException { if (isFull()) throw new BufferFullException(); doPut(v); } public synchronized V take() throws BufferEmptyException { if (isEmpty()) throw new BufferEmptyException(); return doTake(); } }   1817BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers Listing14.4.ClientLogicforCallingGrumpyBoundedBuffer. while (true) { try { V item = buffer.take(); // use item break; } catch (BufferEmptyException e) { Thread.sleep(SLEEP_GRANULARITY); } } Avariantofthisapproachistoreturnanerrorvaluewhenthebufferisinthewrongstate.Thisisaminorimprovement inthatitdoesn'tabusetheexceptionmechanismbythrowinganexceptionthatreallymeans"sorry,tryagain",butit doesnotaddressthefundamentalproblem:thatcallersmustdealwithpreconditionfailuresthemselves.[2] [2]QueueoffersbothoftheseoptionsͲpollreturnsnullifthequeueisempty,andremovethrowsanexceptionͲbutQueueisnotintended foruseinproducerͲconsumerdesigns.BlockingQueue,whoseoperationsblockuntilthequeueisintherightstatetoproceed,isabetter choicewhenproducersandconsumerswillexecuteconcurrently. The client code in Listing 14.4 is not the only way to implement the retry logic. The caller could retry the take immediately,withoutsleepingͲanapproachknownasbusywaitingorspinwaiting.Thiscouldconsumequitealotof CPUtimeifthebufferstatedoesnotchangeforawhile.Ontheotherhand,ifthecallerdecidestosleepsoasnotto consumesomuchCPUtime,itcouldeasily"oversleep"ifthebufferstatechangesshortlyafterthecalltosleep.Sothe clientcodeisleftwiththechoicebetweenthepoorCPUusageofspinningandthepoorresponsivenessofsleeping. (SomewherebetweenbusywaitingandsleepingwouldbecallingThread.yieldineachiteration,whichisahinttothe schedulerthatthiswouldbeareasonabletimetoletanotherthreadrun.Ifyouarewaitingforanotherthreadtodo something,thatsomethingmighthappenfasterifyouyieldtheprocessorratherthanconsumingyourfullscheduling quantum.) 14.1.2.Example:CrudeBlockingbyPollingandSleeping SleepyBoundedBufferinListing14.5attemptstosparecallerstheinconvenienceofimplementingtheretrylogicon eachcallbyencapsulatingthesamecrude"pollandsleep"retrymechanismwithintheputandtakeoperations.Ifthe bufferisempty,takesleepsuntilanotherthreadputssomedataintothebuffer;ifthebufferisfull,putsleepsuntil another thread makes room by removing some data. This approach encapsulates precondition management and simplifiesusingthebufferͲdefinitelyastepintherightdirection. TheimplementationofSleepyBoundedBufferismorecomplicatedthanthepreviousattempt.[3]Thebuffercodemust testtheappropriatestateconditionwiththebufferlockheld,becausethevariablesthatrepresentthestatecondition areguardedbythebufferlock.Ifthetestfails,theexecutingthreadsleepsforawhile,firstreleasingthelocksoother threadscanaccessthebuffer.[4]Oncethethreadwakesup,itreacquiresthelockandtriesagain,alternatingbetween sleepingandtestingthestateconditionuntiltheoperationcanproceed. [3]WewillspareyouthedetailsofSnowWhite'sotherfiveboundedbufferimplementations,especiallySneezyBoundedBuffer. [4]Itisusuallyabadideaforathreadtogotosleeporotherwiseblockwithalockheld,butinthiscaseisevenworsebecausethedesired condition(bufferisfull/empty)canneverbecometrueifthelockisnotreleased! Fromtheperspectiveofthecaller,thisworksnicelyͲiftheoperationcanproceedimmediately,itdoes,andotherwiseit blocksͲandthecallerneednotdealwiththemechanicsoffailureandretry.Choosingthesleepgranularityisatradeoff betweenresponsivenessandCPUusage;thesmallerthesleepgranularity,themoreresponsive,butalsothemoreCPU resourcesconsumed.Figure14.1showshowsleepgranularitycanaffectresponsiveness:theremaybeadelaybetween whenbufferspacebecomesavailableandwhenthethreadwakesupandchecksagain. Figure14.1.ThreadOversleepingBecausetheConditionBecameTrueJustAfterItWenttoSleep.    182 JavaConcurrencyInPractice Listing14.5.BoundedBufferUsingCrudeBlocking. @ThreadSafe public class SleepyBoundedBuffer extends BaseBoundedBuffer { public SleepyBoundedBuffer(int size) { super(size); } public void put(V v) throws InterruptedException { while (true) { synchronized (this) { if (!isFull()) { doPut(v); return; } } Thread.sleep(SLEEP_GRANULARITY); } } public V take() throws InterruptedException { while (true) { synchronized (this) { if (!isEmpty()) return doTake(); } Thread.sleep(SLEEP_GRANULARITY); } } } SleepyBoundedBufferalsocreatesanotherrequirementforthecallerͲdealingwithInterruptedException.Whena methodblockswaitingforaconditiontobecometrue,thepolitethingtodoistoprovideacancellationmechanism(see Chapter7).LikemostwellͲbehaved blockinglibrarymethods, SleepyBounded-Buffersupportscancellationthrough interruption,returningearlyandthrowingInterruptedExceptionifinterrupted. Theseattemptstosynthesizeablockingoperationfrompollingandsleepingwerefairlypainful.Itwouldbenicetohave awayofsuspendingathreadbutensuringthatitisawakenedpromptlywhenacertaincondition(suchasthebuffer beingnolongerfull)becomestrue.Thisisexactlywhatconditionqueuesdo. 14.1.3.ConditionQueuestotheRescue Conditionqueuesarelikethe"toastisready"bellonyourtoaster.Ifyouarelisteningforit,youarenotifiedpromptly whenyourtoastisreadyandcandropwhatyouaredoing(ornot,maybeyouwanttofinishthenewspaperfirst)and getyourtoast.Ifyouarenotlisteningforit(perhapsyouwentoutsideto get thenewspaper),you could missthe notification,butonreturntothekitchenyoucanobservethestateofthetoasterandeitherretrievethetoastifitis finishedorstartlisteningforthebellagainifitisnot. AconditionqueuegetsitsnamebecauseitgivesagroupofthreadsͲcalledthewaitsetͲawaytowaitforaspecific conditiontobecometrue.Unliketypicalqueuesinwhichtheelementsaredataitems,theelementsofacondition queuearethethreadswaitingforthecondition. JustaseachJavaobjectcanactasalock,eachobjectcanalsoactasaconditionqueue,andthewait,notify,and notifyAllmethodsinObjectconstitutetheAPIforintrinsicconditionqueues.Anobject'sintrinsiclockanditsintrinsic conditionqueuearerelated:inordertocallanyoftheconditionqueuemethodsonobjectX,youmustholdthelockon X.ThisisbecausethemechanismforwaitingforstateͲbasedconditionsisnecessarilytightlyboundtothemechanism forpreservingstateconsistency:youcannotwaitforaconditionunlessyoucanexaminethestate,andyoucannot releaseanotherthreadfromaconditionwaitunlessyoucanmodifythestate. Object.waitatomicallyreleasesthelockandaskstheOStosuspendthecurrentthread,allowingotherthreadsto acquirethelockandthereforemodifytheobjectstate.Uponwaking,itreacquiresthelockbeforereturning.Intuitively, calling wait means "I want to go to sleep, but wake me when something interesting happens", and calling the notificationmethodsmeans"somethinginterestinghappened". BoundedBuffer in Listing 14.6 implements a bounded buffer using wait and notifyAll. This is simpler than the sleepingversion,andisbothmoreefficient(wakinguplessfrequentlyifthebufferstatedoesnotchange)andmore responsive(wakinguppromptlywhenaninterestingstatechangehappens).Thisisabigimprovement,butnotethat the introduction of condition queues didn't change the semantics compared to the sleeping version. It is simply an optimization in several dimensions: CPU efficiency, contextͲswitch overhead, and responsiveness. Condition queues  1837BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers don'tletyoudoanythingyoucan'tdowithsleepingandpolling[5],buttheymakeitaloteasierandmoreefficientto expressandmanagestatedependence. [5]Thisisnotquitetrue;afairconditionqueuecanguaranteetherelativeorderinwhichthreadsarereleasedfromthewaitset.Intrinsiccondition queues,likeintrinsiclocks,donotofferfairqueuing;explicitConditionsofferachoiceoffairornonͲfairqueuing. Listing14.6.BoundedBufferUsingConditionQueues. @ThreadSafe public class BoundedBuffer extends BaseBoundedBuffer { // CONDITION PREDICATE: not-full (!isFull()) // CONDITION PREDICATE: not-empty (!isEmpty()) public BoundedBuffer(int size) { super(size); } // BLOCKS-UNTIL: not-full public synchronized void put(V v) throws InterruptedException { while (isFull()) wait(); doPut(v); notifyAll(); } // BLOCKS-UNTIL: not-empty public synchronized V take() throws InterruptedException { while (isEmpty()) wait(); V v = doTake(); notifyAll(); return v; } } BoundedBufferisfinallygoodenoughtouseͲitiseasytouseandmanagesstatedependencesensibly.[6]Aproduction versionshouldalsoincludetimedversionsofputand take,sothatblockingoperationscantimeoutiftheycannot completewithinatimebudget.ThetimedversionofObject.waitmakesthiseasytoimplement. [6]ConditionBoundedBufferinSection14.3isevenbetter:itismoreefficientbecauseitcanusesinglenotificationinsteadofnotifyAll. 14.2.UsingConditionQueues ConditionqueuesmakeiteasiertobuildefficientandresponsivestateͲdependentclasses,buttheyarestilleasytouse incorrectly;therearealotofrulesregardingtheirproperusethatarenotenforcedbythecompilerorplatform.(Thisis one of the reasons to build on top of classes like LinkedBlockingQueue, CountDown-Latch, Semaphore, and FutureTaskwhenyoucan;ifyoucangetawaywithit,itisaloteasier.) 14.2.1.TheConditionPredicate Thekeytousingconditionqueuescorrectlyisidentifyingtheconditionpredicatesthattheobjectmaywaitfor.Itisthe conditionpredicatethatcausesmuchoftheconfusionsurroundingwaitandnotify,becauseithasnoinstantiationin theAPIandnothingineitherthelanguagespecificationortheJVMimplementationensuresitscorrectuse.Infact,itis notmentioneddirectlyatallinthelanguagespecificationortheJavadoc.Butwithoutit,conditionwaitswouldnot work. TheconditionpredicateisthepreconditionthatmakesanoperationstateͲdependentinthefirstplace.Inabounded buffer,takecanproceedonlyifthebufferisnotempty;otherwiseitmustwait.Fortake,theconditionpredicateis"the bufferisnotempty",which take musttestforbeforeproceeding.Similarly,theconditionpredicatefor putis"the buffer is not full". Condition predicates are expressions constructed from the state variables of the class; BaseBoundedBuffer tests for "buffer not empty" by comparing count to zero, and tests for "buffer not full" by comparingcounttothebuffersize. Documenttheconditionpredicate(s)associatedwithaconditionqueueandtheoperationsthatwaitonthem. ThereisanimportantthreeͲwayrelationshipinaconditionwaitinvolvinglocking,thewaitmethod,andacondition predicate.Theconditionpredicateinvolvesstatevariables,andthestatevariablesareguardedbyalock,sobefore testingtheconditionpredicate,wemustholdthatlock.Thelockobjectandtheconditionqueueobject(theobjecton whichwaitandnotifyareinvoked)mustalsobethesameobject. InBoundedBuffer,thebufferstateisguardedbythebufferlockandthebufferobjectisusedastheconditionqueue. Thetakemethodacquiresthebufferlockandthenteststheconditionpredicate(thatthebufferisnonempty).Ifthe  184 JavaConcurrencyInPractice bufferisindeednonempty,itremovesthefirstelement,whichitcandobecauseitstillholdsthelockguardingthe bufferstate. Iftheconditionpredicateisnottrue(thebufferisempty),takemustwaituntilanotherthreadputsanobjectinthe buffer.Itdoesthisbycalling waitonthebuffer'sintrinsicconditionqueue,whichrequiresholdingthelockonthe condition queue object. As careful design would have it, take already holds that lock, which it needed to test the conditionpredicate(andiftheconditionpredicatewastrue,tomodifythebufferstateinthesameatomicoperation). Thewaitmethodreleasesthelock,blocksthecurrentthread,andwaitsuntilthespecifiedtimeoutexpires,thethread isinterrupted,orthethreadisawakenedbyanotification.Afterthethreadwakesup,waitreacquiresthelockbefore returning.Athreadwakingupfromwaitgetsnospecialpriorityinreacquiringthelock;itcontendsforthelockjustlike anyotherthreadattemptingtoenterasynchronizedblock. Everycalltowaitisimplicitlyassociatedwithaspecificconditionpredicate.Whencallingwaitregardingaparticular conditionpredicate,thecallermustalreadyholdthelockassociatedwiththeconditionqueue,andthatlockmustalso guardthestatevariablesfromwhichtheconditionpredicateiscomposed. 14.2.2.WakingUpTooSoon AsifthethreeͲwayrelationshipamongthelock,theconditionpredicate,andtheconditionqueuewerenotcomplicated enough,thatwaitreturnsdoesnotnecessarilymeanthattheconditionpredicatethethreadiswaitingforhasbecome true. Asingleintrinsicconditionqueuemaybeusedwithmorethanoneconditionpredicate.Whenyourthreadisawakened becausesomeonecallednotifyAll,thatdoesn'tmeanthattheconditionpredicateyouwerewaitingforisnowtrue. (Thisislikehavingyourtoasterandcoffeemakershareasinglebell;whenitrings,youstillhavetolooktoseewhich deviceraisedthesignal.)[7]Additionally,waitisevenallowedtoreturn"spuriously”notinresponsetoanythreadcalling notify.[8] [7]ThissituationactuallydescribesTim'skitchenprettywell;somanydevicesbeepthatwhenyouhearone,youhavetoinspectthetoaster,the microwave,thecoffeemaker,andseveralotherstodeterminethecauseofthesignal. [8]Topushthebreakfastanalogywaytoofar,thisislikeatoasterwithalooseconnectionthatmakesthebellgooffwhenthetoastisreadybut alsosometimeswhenitisnotready. WhencontrolreͲentersthecodecallingwait,ithasreacquiredthelockassociatedwiththeconditionqueue.Isthe conditionpredicatenowtrue?Maybe.ItmighthavebeentrueatthetimethenotifyingthreadcallednotifyAll,but could have become false again by the time you reacquire the lock. Other threads may have acquired the lock and changedtheobject'sstatebetweenwhenyourthreadwasawakenedandwhenwaitreacquiredthelock.Ormaybeit hasn'tbeentrueatallsinceyoucalledwait.Youdon'tknowwhyanotherthreadcallednotifyornotifyAll;maybeit wasbecauseanotherconditionpredicateassociatedwiththesameconditionqueuebecametrue.Multiplecondition predicatesperconditionqueuearequitecommonͲBoundedBufferusesthesameconditionqueueforboththe"not full"and"notempty"predicates.[9] [9]Itisactuallypossibleforthreadstobewaitingforboth"notfull"and"notempty"atthesametime!Thiscanhappenwhenthenumberof producers/consumersexceedsthebuffercapacity. Forallthesereasons,whenyouwakeupfromwaityoumusttesttheconditionpredicateagain,andgobacktowaiting (orfail)ifitisnotyettrue.Sinceyoucanwakeuprepeatedlywithoutyourconditionpredicatebeingtrue,youmust thereforealwayscallwaitfromwithinaloop,testingtheconditionpredicateineachiteration.Thecanonicalformfora conditionwaitisshowninListing14.7. Listing14.7.CanonicalFormforStateǦdependentMethods. void stateDependentMethod() throws InterruptedException { // condition predicate must be guarded by lock synchronized(lock) { while (!conditionPredicate()) lock.wait(); // object is now in desired state } }     1857BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers Whenusingconditionwaits(Object.waitorCondition.await): x Alwayshaveaconditionpredicatesometestofobjectstatethatmustholdbeforeproceeding; x Alwaystesttheconditionpredicatebeforecallingwait,andagainafterreturningfromwait; x Alwayscallwaitinaloop; x Ensure that the state variables making up the condition predicate are guarded by the lock associated with the conditionqueue; x Holdthelockassociatedwiththetheconditionqueuewhencallingwait,notify,ornotifyAll;and x Donotreleasethelockaftercheckingtheconditionpredicatebutbeforeactingonit. 14.2.3.MissedSignals Chapter10discussedlivenessfailuressuchasdeadlockandlivelock.Anotherformoflivenessfailureismissedsignals.A missed signal occurs when a thread must wait for a specific condition that is already true, but fails to check the conditionpredicatebeforewaiting.Nowthethreadiswaitingtobenotifiedofaneventthathasalreadyoccurred.This islikestartingthetoast,goingouttogetthenewspaper,havingthebellgooffwhileyouareoutside,andthensitting down at the kitchen table waiting for the toast bell. You could wait a long timepotentially forever.[10] Unlike the marmaladeforyourtoast,notificationisnot"sticky"ifthreadAnotifiesonaconditionqueueandthreadBsubsequently waitsonthatsameconditionqueue,BdoesnotimmediatelywakeupanothernotificationisrequiredtowakeB.Missed signalsaretheresultofcodingerrorslikethosewarnedagainstinthelistabove,suchasfailingtotestthecondition predicatebeforecallingwait.IfyoustructureyourconditionwaitsasinListing14.7,youwillnothaveproblemswith missedsignals. [10]Inordertoemergefromthiswait,someoneelsewouldhavetomaketoast,butthiswilljustmakemattersworse;whenthebellrings,youwill thenhaveadisagreementabouttoastownership. 14.2.4.Notification Sofar,we'vedescribedhalfofwhatgoesoninaconditionwait:waiting.Theotherhalfisnotification.Inabounded buffer,takeblocksifcalledwhenthebufferisempty.Inorderfortaketounblockwhenthebufferbecomesnonempty, we must ensure that every code path in which the buffer could become nonempty performs a notification. In BoundedBuffer,thereisonlyonesuchplaceafteraput.SoputcallsnotifyAllaftersuccessfullyaddinganobjectto thebuffer.Similarly,takecallsnotifyAllafterremovinganelementtoindicatethatthebuffermaynolongerbefull, incaseanythreadsarewaitingonthe"notfull"condition. Whenever you wait on a condition, make sure that someone will perform a notification whenever the condition predicatebecomestrue. TherearetwonotificationmethodsintheconditionqueueAPInotifyandnotifyAll.Tocalleither,youmustholdthe lockassociatedwiththeconditionqueueobject.CallingnotifycausestheJVMtoselectonethreadwaitingonthat conditionqueuetowakeup;callingnotifyAllwakesupallthethreadswaitingonthatconditionqueue.Becauseyou mustholdthelockontheconditionqueueobjectwhencallingnotifyornotifyAll,andwaitingthreadscannotreturn fromwaitwithoutreacquiringthelock,thenotifyingthreadshouldreleasethelockquicklytoensurethatthewaiting threadsareunblockedassoonaspossible. Becausemultiplethreadscouldbewaitingonthesameconditionqueuefordifferentconditionpredicates,usingnotify instead of notifyAll can be dangerous, primarily because single notification is prone to a problem akin to missed signals. BoundedBufferprovidesagoodillustrationofwhynotifyAllshouldbepreferredtosinglenotifyinmostcases.The conditionqueueisusedfortwodifferentconditionpredicates:"notfull"and"notempty".SupposethreadAwaitsona conditionqueueforpredicatePA,whilethreadBwaitsonthesameconditionqueueforpredicatePB.Now,supposePB becomestrueandthreadCperformsasingle notify:theJVMwillwakeuponethreadofitsownchoosing.IfAis chosen, it will wake up, see that PA is not yet true, and go back to waiting. Meanwhile, B, which could now make progress,doesnotwakeup.Thisisnotexactlyamissedsignalit'smoreofa"hijackedsignal"buttheproblemisthe same:athreadiswaitingforasignalthathas(orshouldhave)alreadyoccurred. SinglenotifycanbeusedinsteadofnotifyAllonlywhenbothofthefollowingconditionshold: x Uniformwaiters.Onlyoneconditionpredicateisassociatedwiththeconditionqueue,andeachthreadexecutesthe samelogicuponreturningfromwait;and x OneͲin,oneͲout.Anotificationontheconditionvariableenablesatmostonethreadtoproceed.  186 JavaConcurrencyInPractice BoundedBuffermeetstheoneͲin,oneͲoutrequirement,butdoesnotmeettheuniformwaitersrequirementbecause waitingthreadsmightbewaitingforeitherthe"notfull"and"notempty"condition.A"startinggate"latchlikethat usedinTestHarnessonpage96,inwhichasingleeventreleasesasetofthreads,doesnotmeettheoneͲin,oneͲout requirementbecauseopeningthestartinggateletsmultiplethreadsproceed. Most classes don't meet these requirements, so the prevailing wisdom is to use notifyAll in preference to single notify. While this may be inefficient, it is much easier to ensure that your classes behave correctly when using notifyAllinsteadofnotify. This"prevailingwisdom"makessomepeopleuncomfortable,andforgoodreason.Using notifyAllwhenonlyone threadcanmakeprogressisinefficientsometimesalittle,sometimesgrosslyso.Iftenthreadsarewaitingonacondition queue,callingnotifyAllcauseseachofthemtowakeupandcontendforthelock;thenmostorallofthemwillgo rightbacktosleep.Thismeansalotofcontextswitchesandalotofcontendedlockacquisitionsforeacheventthat enables(maybe)asinglethreadtomakeprogress.(Intheworstcase,usingnotify-AllresultsinO(n2)wakeupswhere n would suffice.) This is another situation where performance concerns support one approach and safety concerns supporttheother. Thenotificationdoneby putand takein BoundedBufferisconservative:anotification isperformedeverytime an objectisputintoorremovedfromthebuffer.Thiscouldbeoptimizedbyobservingthatathreadcanbereleasedfroma waitonlyifthebuffergoesfromemptytonotemptyorfromfulltonotfull,andnotifyingonlyifaputortakeeffected one of these state transitions. This is called conditional notification. While conditional notification can improve performance,itistrickytogetright(andalsocomplicatestheimplementationofsubclasses)andsoshouldbeused carefully.Listing14.8illustratesusingconditionalnotificationinBoundedBuffer.put. Singlenotificationandconditionalnotificationareoptimizations.Asalways,followtheprinciple"Firstmakeitright,and thenmakeitfastifitisnotalreadyfastenough"whenusingtheseoptimizations;itiseasytointroducestrangeliveness failuresbyapplyingthemincorrectly. Listing14.8.UsingConditionalNotificationinBoundedBuffer.put. public synchronized void put(V v) throws InterruptedException { while (isFull()) wait(); boolean wasEmpty = isEmpty(); doPut(v); if (wasEmpty) notifyAll(); } 14.2.5.Example:AGateClass ThestartinggatelatchinTestHarnessonpage96wasconstructedwithaninitialcountofone,creatingabinarylatch: onewithtwostates,theinitialstateandtheterminalstate.Thelatchpreventsthreadsfrompassingthestartinggate untilitisopened,atwhichpointallthethreadscanpassthrough.Whilethislatchingmechanismisoftenexactlywhatis needed,sometimesitisadrawbackthatagateconstructedinthismannercannotbereclosedonceopened. ItiseasytodeveloparecloseableThreadGateclassusingconditionwaits,asshowninListing14.9.ThreadGateletsthe gatebeopenedandclosed,providinganawaitmethodthatblocksuntilthegateisopened.The openmethoduses notifyAllbecausethesemanticsofthisclassfailthe"oneͲin,oneͲout"testforsinglenotification. TheconditionpredicateusedbyawaitismorecomplicatedthansimplytestingisOpen.ThisisneededbecauseifN threadsarewaitingatthegateatthetimeitisopened,theyshouldallbeallowedtoproceed.But,ifthegateisopened andclosedinrapidsuccession,allthreadsmightnotbereleasedifawaitexaminesonlyisOpen:bythetimeallthe threads receive the notification, reacquire the lock, and emerge from wait, the gate may have closed again. So ThreadGate uses a somewhat more complicated condition predicate: every time the gate is closed, a "generation" counterisincremented,andathreadmaypassawaitifthegateisopennoworifthegatehasopenedsincethisthread arrivedatthegate. Since ThreadGateonlysupportswaitingforthegatetoopen,itperformsnotificationonlyin open;tosupportboth "waitforopen"and"waitforclose"operations,itwouldhavetonotifyinbothopenandclose.Thisillustrateswhy stateͲdependent classes can be fragile to maintainthe addition of a new statedependent operation may require modifyingmanycodepathsthatmodifytheobjectstatesothattheappropriatenotificationscanbeperformed.  1877BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers 14.2.6.SubclassSafetyIssues Usingconditionalorsinglenotificationintroducesconstraintsthatcancomplicatesubclassing[CPJ3.3.3.3].Ifyouwant tosupportsubclassingatall,youmuststructureyourclasssosubclassescanaddtheappropriatenotificationonbehalf ofthebaseclassifitissubclassedinawaythatviolatesoneoftherequirementsforsingleorconditionalnotification. Listing14.9.RecloseableGateUsingWaitandNotifyall. @ThreadSafe public class ThreadGate { // CONDITION-PREDICATE: opened-since(n) (isOpen || generation>n) @GuardedBy("this") private boolean isOpen; @GuardedBy("this") private int generation; public synchronized void close() { isOpen = false; } public synchronized void open() { ++generation; isOpen = true; notifyAll(); } // BLOCKS-UNTIL: opened-since(generation on entry) public synchronized void await() throws InterruptedException { int arrivalGeneration = generation; while (!isOpen && arrivalGeneration == generation) wait(); } } AstateͲdependentclassshouldeitherfullyexpose(anddocument)itswaitingandnotificationprotocolstosubclasses, orpreventsubclassesfromparticipatinginthematall.(Thisisanextensionof"designanddocumentforinheritance,or elseprohibitit"[EJItem15].)Attheveryleast,designingastateͲdependentclassforinheritancerequiresexposingthe conditionqueuesandlocksanddocumentingtheconditionpredicatesandsynchronizationpolicy;itmayalsorequire exposingtheunderlyingstatevariables.(TheworstthingastateͲdependentclasscandoisexposeitsstatetosubclasses but not document its protocols for waiting and notification; this is like a class exposing its state variables but not documentingitsinvariants.) One option for doing this is to effectively prohibit subclassing, either by making the class final or by hiding the conditionqueues,locks,andstatevariablesfromsubclasses.Otherwise,ifthesubclassdoessomethingtoundermine thewaythebaseclassusesnotify,itneedstobeabletorepairthedamage.Consideranunboundedblockingstackin which the pop operation blocks if the stack is empty but the push operation can always proceed. This meets the requirements for single notification. If this class uses single notification and a subclass adds a blocking "pop two consecutive elements"method,therearenowtwoclassesofwaiters:thosewaiting topoponeelementandthose waitingtopoptwo.Butifthebaseclassexposestheconditionqueueanddocumentsitsprotocolsforusingit,the subclasscanoverridethepushmethodtoperformanotifyAll,restoringsafety. 14.2.7.EncapsulatingConditionQueues Itisgenerallybesttoencapsulatetheconditionqueuesothatitisnotaccessibleoutsidetheclasshierarchyinwhichitis used.Otherwise,callersmightbetemptedtothinktheyunderstandyourprotocolsforwaitingandnotificationanduse theminamannerinconsistentwithyourdesign.(Itisimpossibletoenforcetheuniformwaitersrequirementforsingle notificationunlesstheconditionqueueobjectisinaccessibletocodeyoudonotcontrol;ifaliencodemistakenlywaits onyourconditionqueue,thiscouldsubvertyournotificationprotocolandcauseahijackedsignal.) Unfortunately,thisadviceͲtoencapsulateobjectsusedasconditionqueuesͲisnotconsistentwiththemostcommon design pattern for threadͲsafe classes, in which an object's intrinsic lock is used to guard its state. BoundedBuffer illustratesthiscommonidiom,wherethebufferobjectitselfisthelockandconditionqueue.However,BoundedBuffer couldbeeasilyrestructuredtouseaprivatelockobjectandconditionqueue;theonlydifferencewouldbethatitwould nolongersupportanyformofclientͲsidelocking. 14.2.8.EntryandExitProtocols Wellings(Wellings,2004)characterizestheproperuseofwaitandnotifyintermsofentryandexitprotocols.Foreach stateͲdependent operation and for each operation that modifies state on which another operation has a state dependency, you should define and document an entry and exit protocol. The entry protocol is the operation's conditionpredicate;theexitprotocolinvolvesexamininganystatevariablesthathavebeenchangedbytheoperation  188 JavaConcurrencyInPractice toseeiftheymighthavecausedsomeotherconditionpredicatetobecometrue,andifso,notifyingontheassociated conditionqueue. AbstractQueuedSynchronizer,uponwhichmostofthestateͲdependentclassesinjava.util.concurrentarebuilt (seeSection14.4),exploitstheconceptofexitprotocol.Ratherthanlettingsynchronizerclassesperformtheirown notification, it instead requires synchronizer methods to return a value indicating whether its action might have unblockedoneormorewaitingthreads.ThisexplicitAPIrequirementmakesitharderto"forget"tonotifyonsome statetransitions. 14.3.ExplicitConditionObjects AswesawinChapter13,explicitLockscanbeusefulinsomesituationswhereintrinsiclocksaretooinflexible.Justas Lockisageneralizationofintrinsiclocks,Condition(seeListing14.10)isageneralizationofintrinsicconditionqueues. Intrinsicconditionqueueshaveseveraldrawbacks.Eachintrinsiclockcanhaveonlyoneassociatedconditionqueue, whichmeansthatinclasseslikeBoundedBuffermultiplethreadsmightwaitonthesameconditionqueuefordifferent conditionpredicates,andthemostcommonpatternforlockinginvolvesexposingtheconditionqueueobject.Bothof thesefactorsmakeitimpossibletoenforcetheuniformwaiterrequirementforusingnotifyAll.Ifyouwanttowritea concurrentobjectwithmultipleconditionpredicates,oryouwanttoexercisemorecontroloverthevisibilityofthe condition queue, the explicit Lock and Condition classes offer a more flexible alternative to intrinsic locks and conditionqueues. AConditionisassociatedwithasingleLock,justasaconditionqueueisassociatedwithasingleintrinsiclock;tocreate aCondition,callLock.newConditionontheassociatedlock.AndjustasLockoffersaricherfeaturesetthanintrinsic locking,Conditionoffersaricherfeaturesetthanintrinsicconditionqueues:multiplewaitsetsperlock,interruptible anduninterruptibleconditionwaits,deadlineͲbasedwaiting,andachoiceoffairornonfairqueueing. Listing14.10.ConditionInterface. public interface Condition { void await() throws InterruptedException; boolean await(long time, TimeUnit unit) throws InterruptedException; long awaitNanos(long nanosTimeout) throws InterruptedException; void awaitUninterruptibly(); boolean awaitUntil(Date deadline) throws InterruptedException; void signal(); void signalAll(); } Unlikeintrinsicconditionqueues,youcanhaveasmanyConditionobjectsperLockasyouwant.Conditionobjects inheritthefairnesssettingoftheirassociatedLock;forfairlocks,threadsarereleasedfromCondition.awaitinFIFO order. Hazard warning: The equivalents of wait, notify, and notifyAll for Condition objects are await, signal, and signalAll.However,ConditionextendsObject,whichmeansthatitalsohaswaitandnotifymethods.Besureto usetheproperversionsͲawaitandsignal - instead! Listing 14.11 shows yet another bounded buffer implementation, this time using two Conditions, notFull and notEmpty,torepresentexplicitlythe"notfull"and"notempty"conditionpredicates.Whentakeblocksbecausethe bufferisempty,itwaitsonnotEmpty,andputunblocksanythreadsblockedintakebysignalingonnotEmpty. The behavior of ConditionBoundedBuffer is the same as BoundedBuffer, but its use of condition queues is more readableͲitiseasiertoanalyzeaclassthatusesmultipleConditionsthanonethatusesasingleintrinsiccondition queue with multiple condition predicates. By separating the two condition predicates into separate wait sets, Conditionmakesiteasiertomeettherequirementsforsinglenotification.Usingthemoreefficientsignalinsteadof signalAllreducesthenumberofcontextswitchesandlockacquisitionstriggeredbyeachbufferoperation. JustaswithbuiltͲinlocksandconditionqueues,thethreeͲwayrelationshipamongthelock,theconditionpredicate,and theconditionvariablemustalsoholdwhenusingexplicitLocksandConditions.Thevariablesinvolvedinthecondition predicatemustbeguardedbytheLock,andtheLockmustbeheldwhentestingtheconditionpredicateandwhen callingawaitandsignal.[11] [11]ReentrantLock requiresthattheLockbeheldwhencallingsignalor signalAll,but Lock implementationsarepermittedto constructConditionsthatdonothavethisrequirement.  1897BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers Choose between using explicit Conditions and intrinsic condition queues in the same way as you would choose betweenReentrantLockandsynchronized:useConditionifyouneeditsadvancedfeaturessuchasfairqueueingor multiplewaitsetsperlock,andotherwisepreferintrinsicconditionqueues.(IfyoualreadyuseReentrantLockbecause youneeditsadvancedfeatures,thechoiceisalreadymade.) 14.4.AnatomyofaSynchronizer TheinterfacesofReentrantLockandSemaphorehavealotincommon.Bothclassesactasa"gate",allowingonlya limited number of threads to pass at a time; threads arrive at the gate and are allowed through (lock or acquire returnssuccessfully),aremadetowait(lockoracquireblocks),orareturnedaway(tryLockortryAcquirereturns false,indicatingthatthelockorpermitdidnotbecomeavailableinthetimeallowed).Further,bothallowinterruptible, uninterruptible,andtimedacquisitionattempts,andbothallowachoiceoffairornonfairqueueingofwaitingthreads. Given this commonality, you might think that Semaphore was implemented on top of ReentrantLock, or perhaps ReentrantLockwasimplementedasa Semaphorewithonepermit.Thiswouldbeentirelypractical;itisacommon exercisetoprovethatacountingsemaphorecanbeimplementedusingalock(asinSemaphoreOnLockinListing14.12) andthatalockcanbeimplementedusingacountingsemaphore. Inactuality,theyarebothimplementedusingacommonbaseclass,Abstract-QueuedSynchronizer(AQS)asaremany other synchronizers. AQS is a framework for building locks and synchronizers, and a surprisingly broad range of synchronizerscanbebuilteasilyandefficientlyusingit.NotonlyareReentrantLockandSemaphorebuiltusingAQS, butsoareCountDownLatch,ReentrantReadWriteLock,SynchronousQueue,[12]andFutureTask. [12]Java6replacestheAQSͲbasedSynchronousQueuewitha(morescalable)nonͲblockingversion. Listing14.11.BoundedBufferUsingExplicitConditionVariables. @ThreadSafe public class ConditionBoundedBuffer { protected final Lock lock = new ReentrantLock(); // CONDITION PREDICATE: notFull (count < items.length) private final Condition notFull = lock.newCondition(); // CONDITION PREDICATE: notEmpty (count > 0) private final Condition notEmpty = lock.newCondition(); @GuardedBy("lock") private final T[] items = (T[]) new Object[BUFFER_SIZE]; @GuardedBy("lock") private int tail, head, count; // BLOCKS-UNTIL: notFull public void put(T x) throws InterruptedException { lock.lock(); try { while (count == items.length) notFull.await(); items[tail] = x; if (++tail == items.length) tail = 0; ++count; notEmpty.signal(); } finally { lock.unlock(); } } // BLOCKS-UNTIL: notEmpty public T take() throws InterruptedException { lock.lock(); try { while (count == 0) notEmpty.await(); T x = items[head]; items[head] = null; if (++head == items.length) head = 0; --count; notFull.signal(); return x; } finally { lock.unlock(); } } }   190 JavaConcurrencyInPractice Listing14.12.CountingSemaphoreImplementedUsingLock. // Not really how java.util.concurrent.Semaphore is implemented @ThreadSafe public class SemaphoreOnLock { private final Lock lock = new ReentrantLock(); // CONDITION PREDICATE: permitsAvailable (permits > 0) private final Condition permitsAvailable = lock.newCondition(); @GuardedBy("lock") private int permits; SemaphoreOnLock(int initialPermits) { lock.lock(); try { permits = initialPermits; } finally { lock.unlock(); } } // BLOCKS-UNTIL: permitsAvailable public void acquire() throws InterruptedException { lock.lock(); try { while (permits <= 0) permitsAvailable.await(); --permits; } finally { lock.unlock(); } } public void release() { lock.lock(); try { ++permits; permitsAvailable.signal(); } finally { lock.unlock(); } } } AQShandlesmanyofthedetailsofimplementingasynchronizer,suchasFIFOqueuingofwaitingthreads.Individual synchronizerscandefineflexiblecriteriaforwhetherathreadshouldbeallowedtopassorberequiredtowait. Using AQS to build synchronizers offers several benefits. Not only does it substantially reduce the implementation effort,butyoualsoneedn'tpayformultiplepointsofcontention,asyouwouldwhenconstructingonesynchronizeron topofanother.InSemaphoreOnLock,acquiringapermithastwoplaceswhereitmightblockͲonceatthelockguarding thesemaphorestate,andthenagainifapermitisnotavailable.SynchronizersbuiltwithAQShaveonlyonepointwhere theymightblock,reducingcontextͲswitchoverheadandimprovingthroughput.AQSwasdesignedforscalability,andall thesynchronizersinjava.util.concurrentthatarebuiltwithAQSbenefitfromthis. 14.5.AbstractQueuedSynchronizer MostdeveloperswillprobablyneveruseAQSdirectly;thestandardsetofsynchronizerscoversafairlywiderangeof situations.Butseeinghowthestandardsynchronizersareimplementedcanhelpclarifyhowtheywork. ThebasicoperationsthatanAQSͲbasedsynchronizerperformsaresomevariantsofacquireandrelease.Acquisitionis the stateͲdependent operation and can always block. With a lock or semaphore, the meaning of acquire is straightforwardͲacquirethelockorapermitͲandthecallermayhavetowaituntilthesynchronizerisinastatewhere thatcanhappen.WithCountDownLatch,acquiremeans"waituntilthelatchhasreacheditsterminalstate",andwith FutureTask,itmeans"waituntilthetaskhascompleted".Releaseisnotablockingoperation;areleasemayallow threadsblockedinacquiretoproceed. ForaclasstobestateͲdependent,itmusthavesomestate.AQStakesonthetaskofmanagingsomeofthestateforthe synchronizer class: it manages a single integer of state information that can be manipulated through the protected getState,setState,andcompareAndSetStatemethods.Thiscanbeusedtorepresentarbitrarystate;forexample, ReentrantLockusesittorepresentthecountoftimestheowningthreadhasacquiredthelock,Semaphoreusesitto representthenumberofpermitsremaining,andFutureTaskusesittorepresentthestateofthetask(notyetstarted, running, completed, cancelled). Synchronizers can also manage additional state variables themselves; for example, ReentrantLock keeps track of the current lock owner so it can distinguish between reentrant and contended lockͲ acquisitionrequests.  1917BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers AcquisitionandreleaseinAQStaketheformsshowninListing14.13.Dependingonthesynchronizer,acquisitionmight beexclusive,aswithReentrant-Lock,ornonexclusive,aswithSemaphoreandCountDownLatch.Anacquireoperation hastwoparts.First,thesynchronizerdecideswhetherthecurrentstatepermitsacquisition;ifso,thethreadisallowed to proceed, and if not, the acquire blocks or fails. This decision is determined by the synchronizer semantics; for example,acquiringalockcansucceedifthelockisunheld,andacquiringalatchcansucceedifthelatchisinitsterminal state. Thesecondpartinvolvespossiblyupdatingthesynchronizerstate;onethreadacquiringthesynchronizercanaffect whetherotherthreadscanacquireit.Forexample,acquiringalockchangesthelockstatefrom"unheld"to"held",and acquiringapermitfromaSemaphorereducesthenumberofpermitsleft.Ontheotherhand,theacquisitionofalatch byonethreaddoesnotaffectwhetherotherthreadscanacquireit,soacquiringalatchdoesnotchangeitsstate. Listing14.13.CanonicalFormsforAcquisitionandReleaseinAQS. boolean acquire() throws InterruptedException { while (state does not permit acquire) { if (blocking acquisition requested) { enqueue current thread if not already queued block current thread } else return failure } possibly update synchronization state dequeue thread if it was queued return success } void release() { update synchronization state if (new state may permit a blocked thread to acquire) unblock one or more queued threads } A synchronizer supporting exclusive acquisition should implement the protected methods TRyAcquire, TRyRelease, and isHeldExclusively, and those supporting shared acquisition should implement tryAcquireShared and TRyReleaseShared.Theacquire,acquireShared,release,andreleaseSharedmethodsinAQScalltheTRyformsof thesemethodsinthesynchronizersubclasstodetermineiftheoperationcanproceed.Thesynchronizersubclasscan use getState, setState, and compareAndSetState to examine and update the state according to its acquire and releasesemantics,andinformsthebaseclassthroughthereturnstatuswhethertheattempttoacquireorreleasethe synchronizer was successful. For example, returning a negative value from TRyAcquireShared indicates acquisition failure;returningzeroindicatesthesynchronizerwasacquiredexclusively;andreturningapositivevalueindicatesthe synchronizerwasacquirednonexclusively.TheTRyReleaseandTRyReleaseSharedmethodsshouldreturntrueifthe releasemayhaveunblockedthreadsattemptingtoacquirethesynchronizer. Tosimplifyimplementationoflocksthatsupportconditionqueues(likeReentrantLock),AQSalsoprovidesmachinery forconstructingconditionvariablesassociatedwithsynchronizers. 14.5.1.ASimpleLatch OneShotLatchinListing14.14isabinarylatchimplementedusingAQS.Ithastwopublicmethods,awaitandsignal, thatcorrespondtoacquisitionandrelease.Initially,thelatchisclosed;anythreadcallingawaitblocksuntilthelatchis opened.Oncethelatchisopenedbyacalltosignal,waitingthreadsarereleasedandthreadsthatsubsequentlyarrive atthelatchwillbeallowedtoproceed.  192 JavaConcurrencyInPractice Listing14.14.BinaryLatchUsingAbstractQueuedSynchronizer. @ThreadSafe public class OneShotLatch { private final Sync sync = new Sync(); public void signal() { sync.releaseShared(0); } public void await() throws InterruptedException { sync.acquireSharedInterruptibly(0); } private class Sync extends AbstractQueuedSynchronizer { protected int tryAcquireShared(int ignored) { // Succeed if latch is open (state == 1), else fail return (getState() == 1) ? 1 : -1; } protected boolean tryReleaseShared(int ignored) { setState(1); // Latch is now open return true; // Other threads may now be able to acquire } } } In OneShotLatch, the AQS state holds the latch stateͲclosed (zero) or open (one). The await method calls acquireSharedInterruptiblyinAQS,whichinturnconsultsthe TRyAcquireSharedmethodin OneShotLatch.The tryAcquire-Sharedimplementationmustreturnavalueindicatingwhetherornotacquisitioncanproceed.Ifthelatch hasbeenpreviouslyopened, tryAcquireSharedreturnssuccess,allowingthethreadtopass;otherwiseitreturnsa value indicating that the acquisition attempt failed. The acquireSharedInterruptibly method interprets failure to meanthatthethreadshouldbeplacedonthequeueofwaitingthreads.Similarly,signalcallsreleaseShared,which causestryReleaseSharedtobeconsulted.TheTRyReleaseSharedimplementationunconditionallysetsthelatchstate toopenandindicates(throughitsreturnvalue)thatthesynchronizerisinafullyreleasedstate.ThiscausesAQStolet allwaitingthreadsattempttoreacquirethesynchronizer,andacquisitionwillnowsucceedbecausetryAcquireShared returnssuccess. OneShotLatchisafullyfunctional,usable,performantsynchronizer,implementedinonlytwentyorsolinesofcode.Of course,itismissingsomeusefulfeatureͲssuchastimedacquisitionortheabilitytoinspectthelatchstatebuttheseare easy to implement as well, since AQS provides timed versions of the acquisition methods and utility methods for commoninspectionoperations. OneShotLatchcouldhavebeenimplementedbyextendingAQSratherthandelegatingtoit,butthisisundesirablefor severalreasons[EJItem14].Doingsowouldunderminethesimple(twoͲmethod)interfaceofOneShotLatch,andwhile thepublicmethodsofAQSwon'tallowcallerstocorruptthelatchstate,callerscouldeasilyusethemincorrectly.None ofthesynchronizersinjava.util.concurrentextendsAQSdirectlyͲtheyalldelegatetoprivateinnersubclassesof AQSinstead. 14.6.AQSinJava.util.concurrentSynchronizerClasses Many of the blocking classes in java.util.concurrent, such as ReentrantLock, Semaphore, ReentrantReadWriteLock,CountDownLatch,SynchronousQueue,andFutureTask,arebuiltusingAQS.Withoutgetting toodeeplyintothedetails(thesourcecodeispartoftheJDKdownload[13]),let'stakeaquicklookathoweachofthese classesusesAQS. [13]Orwithfewerlicensingrestrictionsathttp://gee.cs.oswego.edu/dl/concurrency-interest. 14.6.1.ReentrantLock ReentrantLock supports only exclusive acquisition, so it implements tryAcquire, tryRelease, and isHeldExclusively; tryAcquire for the nonͲfair version is shown in Listing 14.15. ReentrantLock uses the synchronizationstatetoholdthelockacquisitioncount,andmaintainsan ownervariableholdingtheidentityofthe owningthreadthatismodifiedonlywhenthecurrentthreadhasjustacquiredthelockorisjustabouttoreleaseit.[14]In tryRelease,itcheckstheownerfieldtoensurethatthecurrentthreadownsthelockbeforeallowinganunlockto proceed;intryAcquire,itusesthisfieldtodifferentiatebetweenareentrantacquisitionandacontendedacquisition attempt.  1937BPartIV:AdvancedTopics Ͳ 26BChapter14Ͳ BuildingCustomSynchronizers [14]BecausetheprotectedstateͲmanipulationmethodshavethememorysemanticsofavolatilereadorwriteandReentrantLockiscarefulto readtheownerfieldonlyaftercallinggetStateandwriteitonlybeforecallingsetState,ReentrantLockcanpiggybackonthememory semanticsofthesynchronizationstate,andthusavoidfurthersynchronizationͲseeSection16.1.4. Whenathreadattemptstoacquirealock,tryAcquirefirstconsultsthelockstate.Ifitisunheld,ittriestoupdatethe lockstatetoindicatethatitisheld.Becausethestatecouldhavechangedsinceitwasfirstinspectedafewinstructions ago,tryAcquireusescompareAndSetStatetoattempttoatomicallyupdatethestatetoindicatethatthelockisnow heldandconfirmthatthestatehasnotchangedsincelastobserved.(SeethedescriptionofcompareAndSetinSection 15.3.)Ifthelockstateindicatesthatitisalreadyheld,ifthecurrentthreadistheownerofthelock,theacquisition countisincremented;ifthecurrentthreadisnottheownerofthelock,theacquisitionattemptfails. Listing14.15.tryAcquireImplementationFromNonǦfairReentrantLock. protected boolean tryAcquire(int ignored) { final Thread current = Thread.currentThread(); int c = getState(); if (c == 0) { if (compareAndSetState(0, 1)) { owner = current; return true; } } else if (current == owner) { setState(c+1); return true; } return false; } ReentrantLock also takes advantage of AQS's builtͲin support for multiple condition variables and wait sets. Lock.newConditionreturnsanewinstanceofConditionObject,aninnerclassofAQS. 14.6.2.SemaphoreandCountDownLatch SemaphoreusestheAQSsynchronizationstatetoholdthecountofpermitscurrentlyavailable.ThetryAcquireShared method(seeListing14.16)firstcomputesthenumberofpermitsremaining,andiftherearenotenough,returnsavalue indicatingthattheacquirefailed.Ifsufficientpermitsappeartobeleft,itattemptstoatomicallyreducethepermit count using compareAndSetState. If that succeeds (meaning that the permit count had not changed since it last looked),itreturnsavalueindicatingthattheacquiresucceeded.Thereturnvaluealsoencodeswhetherothershared acquisitionattemptsmightsucceed,inwhichcaseotherwaitingthreadswillalsobeunblocked. The while loop terminates either when there are not enough permits or when TRyAcquireShared can atomically updatethepermitcounttoreflectacquisition.WhileanygivencalltocompareAndSetStatemayfailduetocontention withanotherthread(seeSection15.3),causingittoretry,oneofthesetwoterminationcriteriawillbecometruewithin areasonablenumberofretries.Similarly,tryReleaseSharedincreasesthepermitcount,potentiallyunblockingwaiting threads,andretriesuntiltheupdatesucceeds.ThereturnvalueofTRyReleaseSharedindicateswhetherotherthreads mighthavebeenunblockedbytherelease. CountDownLatchusesAQSinasimilarmannertoSemaphore:thesynchronizationstateholdsthecurrentcount.The countDownmethodcalls release,whichcausesthecountertobedecrementedandunblockswaitingthreadsifthe counter has reached zero; await calls acquire, which returns immediately if the counter has reached zero and otherwiseblocks. Listing14.16.tryacquiresharedandtryreleasesharedfromSemaphore. protected int tryAcquireShared(int acquires) { while (true) { int available = getState(); int remaining = available - acquires; if (remaining < 0 || compareAndSetState(available, remaining)) return remaining; } } protected boolean tryReleaseShared(int releases) { while (true) { int p = getState(); if (compareAndSetState(p, p + releases)) return true; } }   194 JavaConcurrencyInPractice 14.6.3.FutureTask Atfirstglance,FutureTaskdoesn'tevenlooklikeasynchronizer.ButFuture.gethassemanticsthatareverysimilarto thatofalatchifsomeevent(thecompletionorcancellationofthetaskrepresentedbytheFutureTask)hasoccurred, thenthreadscanproceed,otherwisetheyarequeueduntilthateventoccurs. FutureTask uses the AQS synchronization state to hold the task statusrunning, completed, or cancelled. It also maintainsadditionalstatevariablestoholdtheresultofthecomputationortheexceptionitthrew.Itfurthermaintains a reference to the thread that is running the computation (if it is currently in the running state), so that it can be interruptedifthetaskiscancelled. 14.6.4.ReentrantReadWriteLock The interface for ReadWriteLock suggests there are two locksa reader lock and a writer lockbut in the AQSͲbased implementation of ReentrantReadWriteLock, a single AQS subclass manages both read and write locking. ReentrantRead-WriteLockuses16bitsofthestateforthewriteͲlockcount,andtheother16bitsforthereadͲlock count.Operationsonthereadlockusethesharedacquireandreleasemethods;operationsonthewritelockusethe exclusiveacquireandreleasemethods. Internally,AQSmaintainsaqueueofwaitingthreads,keepingtrackofwhetherathreadhasrequestedexclusiveor sharedaccess.InReentrantRead-WriteLock,whenthelockbecomesavailable,ifthethreadattheheadofthequeue waslookingforwriteaccessitwillgetit,andifthethreadattheheadofthequeuewaslookingforreadaccess,all queuedthreadsuptothefirstwriterwillgetit.[15] [15]ThismechanismdoesnotpermitthechoiceofareaderͲpreferenceorwriterͲpreferencepolicy,assomereadͲwritelockimplementationsdo. Forthat,eithertheAQSwaitqueuewouldneedtobesomethingotherthanaFIFOqueue,ortwoqueueswouldbeneeded.However,suchastrict orderingpolicyisrarelyneededinpractice;ifthenonfairversionofReentrantReadWriteLockdoesnotofferacceptableliveness,thefair versionusuallyprovidessatisfactoryorderingandguaranteesnonstarvationofreadersandwriters. Summary IfyouneedtoimplementastateͲdependentclassͲonewhosemethodsmustblockifastateͲbasedpreconditiondoes notholdͲthebeststrategyisusuallytobuilduponanexistinglibraryclasssuchas Semaphore, BlockingQueue,or CountDownLatch,asinValueLatchonpage187.However,sometimesexistinglibraryclassesdonotprovideasufficient foundation;inthesecases,youcanbuildyourownsynchronizersusingintrinsicconditionqueues,explicitCondition objects,or AbstractQueuedSynchronizer.Intrinsicconditionqueuesaretightlyboundtointrinsiclocking,sincethe mechanism for managing state dependence is necessarily tied to the mechanism for ensuring state consistency. Similarly, explicit Conditions are tightly bound to explicit Locks, and offer an extended feature set compared to intrinsicconditionqueues,includingmultiplewaitsetsperlock,interruptibleoruninterruptibleconditionwaits,fairor nonfairqueuing,anddeadlineͲbasedwaiting.   1957BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization Chapter15.AtomicVariablesandNonǦblockingSynchronization Many of the classes in java.util.concurrent, such as Semaphore and ConcurrentLinkedQueue, provide better performanceandscalabilitythanalternativesusingsynchronized.Inthischapter,wetakealookattheprimarysource ofthisperformanceboost:atomicvariablesandnonͲblockingsynchronization. MuchoftherecentresearchonconcurrentalgorithmshasfocusedonnonͲblockingalgorithms,whichuselowͲlevel atomic machine instructions such as compareͲandͲswap instead of locks to ensure data integrity under concurrent access.NonͲblockingalgorithmsareusedextensivelyinoperatingsystemsandJVMsforthreadandprocessscheduling, garbagecollection,andtoimplementlocksandotherconcurrentdatastructures. NonͲblockingalgorithmsareconsiderablymorecomplicatedtodesignandimplementthanlockͲbasedalternatives,but theycanoffersignificantscalabilityandlivenessadvantages.Theycoordinateatafinerlevelofgranularityandcan greatly reduce scheduling overhead because they don't block when multiple threads contend for the same data. Further, they are immune to deadlock and other liveness problems. In lockͲbased algorithms, other threads cannot makeprogressifathreadgoestosleeporspinswhileholdingalock,whereasnonͲblockingalgorithmsareimperviousto individualthreadfailures.AsofJava5.0,itispossibletobuildefficientnonͲblockingalgorithmsinJavausingtheatomic variableclassessuchasAtomicIntegerandAtomicReference. Atomicvariablescanalsobeusedas"bettervolatilevariables"evenifyouarenotdevelopingnonͲblockingalgorithms. Atomicvariablesofferthesamememorysemanticsasvolatilevariables,butwithadditionalsupportforatomicupdates Ͳmakingthemidealforcounters,sequencegenerators,andstatisticsgatheringwhileofferingbetterscalabilitythan lockͲbasedalternatives. 15.1.DisadvantagesofLocking Coordinatingaccesstosharedstateusingaconsistentlockingprotocolensuresthatwhicheverthreadholdsthelock guardingasetofvariableshasexclusiveaccesstothosevariables,andthatanychangesmadetothosevariablesare visibletootherthreadsthatsubsequentlyacquirethelock. ModernJVMscanoptimizeuncontendedlockacquisitionandreleasefairlyeffectively,butifmultiplethreadsrequest thelockatthesametimetheJVMenliststhehelpoftheoperatingsystem.Ifitgetstothispoint,someunfortunate threadwillbesuspendedandhavetoberesumedlater.[1]Whenthatthreadisresumed,itmayhavetowaitforother threadstofinishtheirschedulingquantabeforeitisactuallyscheduled.Suspendingandresumingathreadhasalotof overheadandgenerallyentailsalengthyinterruption.ForlockͲbasedclasseswithfineͲgrainedoperations(suchasthe synchronizedcollectionsclasses,wheremostmethodscontainonlyafewoperations),theratioofschedulingoverhead tousefulworkcanbequitehighwhenthelockisfrequentlycontended. [1]AsmartJVMneednotnecessarilysuspendathreadifitcontendsforalock;itcoulduseprofilingdatatodecideadaptivelybetweensuspension andspinlockingbasedonhowlongthelockhasbeenheldduringpreviousacquisitions. Volatile variables are a lighterͲweight synchronization mechanism than locking because they do not involve context switches or thread scheduling. However, volatile variables have some limitations compared to locking: while they provide similar visibility guarantees, they cannot be used to construct atomic compound actions. This means that volatilevariablescannotbeusedwhenonevariabledependsonanother,orwhenthenewvalueofavariabledepends onitsoldvalue.Thislimitswhenvolatilevariablesareappropriate,sincetheycannotbeusedtoreliablyimplement commontoolssuchascountersormutexes.[2] [2]Itistheoreticallypossible,thoughwhollyimpractical,tousethesemanticsofvolatiletoconstructmutexesandothersynchronizers;see (Raynal,1986). For example, while the increment operation (++i) may look like an atomic operation, it is actually three distinct operationsͲfetchthecurrentvalueofthevariable,addonetoit,andthenwritetheupdatedvalueback.Inordertonot loseanupdate,theentirereadͲmodifyͲwriteoperationmustbeatomic.Sofar,theonlywaywe'veseentodothisis withlocking,asinCounteronpage56. Counter is threadͲsafe, and in the presence of little or no contention performs just fine. But under contention, performancesuffersbecauseofcontextͲswitchoverheadandschedulingdelays.Whenlocksareheldsobriefly,being puttosleepisaharshpenaltyforaskingforthelockatthewrongtime. Locking has a few other disadvantages. When a thread is waiting for a lock, it cannot do anything else. If a thread holdingalockisdelayed(duetoapagefault,schedulingdelay,orthelike),thennothreadthatneedsthatlockcan  196 JavaConcurrencyInPractice makeprogress.ThiscanbeaseriousproblemiftheblockedthreadisahighͲprioritythreadbutthethreadholdingthe lock is a lowerͲpriority threadͲa performance hazard known as priority inversion. Even though the higherͲpriority threadshouldhaveprecedence,itmustwaituntilthelockisreleased,andthiseffectivelydowngradesitspriorityto thatofthelowerͲprioritythread.Ifathreadholdingalockispermanentlyblocked(duetoaninfiniteloop,deadlock, livelock,orotherlivenessfailure),anythreadswaitingforthatlockcannevermakeprogress. Even ignoring these hazards, locking is simply a heavyweight mechanism for fineͲgrained operations such as incrementingacounter.ItwouldbenicetohaveafinerͲgrainedtechniqueformanagingcontentionbetweenthreadsͲ somethinglikevolatilevariables,butofferingthepossibilityofatomicupdatesaswell.Happily,modernprocessorsoffer uspreciselysuchamechanism. 15.2.HardwareSupportforConcurrency ExclusivelockingisapessimistictechniqueͲitassumestheworst(ifyoudon'tlockyourdoor,gremlinswillcomeinand rearrange your stuff) and doesn't proceed until you can guarantee, by acquiring the appropriate locks, that other threadswillnotinterfere. For fineͲgrained operations, there is an alternate approach that is often more efficientͲthe optimistic approach, wherebyyouproceedwithanupdate,hopefulthatyoucancompleteitwithoutinterference.Thisapproachrelieson collisiondetectiontodetermineiftherehasbeeninterferencefromotherpartiesduringtheupdate,inwhichcasethe operation fails and can be retried (or not). The optimistic approach is like the old saying, "It is easier to obtain forgivenessthanpermission",where"easier"heremeans"moreefficient". Processorsdesignedformultiprocessoroperationprovidespecialinstructionsformanagingconcurrentaccesstoshared variables. Early processors had atomic testͲandͲset, fetchͲandͲincrement, or swap instructions sufficient for implementingmutexesthatcouldinturnbeusedtoimplementmoresophisticatedconcurrentobjects.Today,nearly everymodernprocessorhassomeformofatomicreadͲmodifyͲwriteinstruction,suchascompareͲandͲswaporloadͲ linked/storeͲconditional.OperatingsystemsandJVMsusetheseinstructionstoimplementlocksandconcurrentdata structures,butuntilJava5.0theyhadnotbeenavailabledirectlytoJavaclasses. 15.2.1.CompareandSwap Theapproachtakenbymostprocessorarchitectures,includingIA32andSparc,istoimplementacompareͲandͲswap (CAS)instruction.(Otherprocessors,suchasPowerPC,implementthesamefunctionalitywithapairofinstructions: loadlinkedandstoreͲconditional.)CAShasthreeoperandsͲamemorylocationVonwhichtooperate,theexpectedold valueA,andthenewvalueB.CASatomicallyupdatesVtothenewvalueB,butonlyifthevalueinVmatchesthe expectedoldvalueA;otherwiseitdoesnothing.Ineithercase,itreturnsthevaluecurrentlyinV.(Thevariantcalled compareͲandͲsetinsteadreturnswhethertheoperationsucceeded.)CASmeans"IthinkVshouldhavethevalueA;ifit does,putBthere,otherwisedon'tchangeitbuttellmeIwaswrong."CASisanoptimistictechniqueͲitproceedswith theupdateinthehopeofsuccess,andcandetectfailureifanotherthreadhasupdatedthevariablesinceitwaslast examined.SimulatedCASinListing15.1illustratesthesemantics(butnottheimplementationorperformance)ofCAS. When multiple threads attempt to update the same variable simultaneously using CAS, one wins and updates the variable'svalue,andtherestlose.Butthelosersarenotpunishedbysuspension,astheycouldbeiftheyfailedto acquirealock;instead,theyaretoldthattheydidn'twintheracethistimebutcantryagain.Becauseathreadthat losesaCASisnotblocked,itcandecidewhetheritwantstotryagain,takesomeotherrecoveryaction,ordonothing.[3] Thisflexibilityeliminatesmanyofthelivenesshazardsassociatedwithlocking(thoughinunusualcasescanintroduce theriskoflivelockͲseeSection10.3.3). [3]DoingnothingmaybeaperfectlysensibleresponsetoafailedCAS;insomenonͲblockingalgorithms,suchasthelinkedqueuealgorithmin Section15.4.2,afailedCASmeansthatsomeoneelsealreadydidtheworkyouwereplanningtodo.  1977BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization Listing15.1.SimulatedCASOperation. @ThreadSafe public class SimulatedCAS { @GuardedBy("this") private int value; public synchronized int get() { return value; } public synchronized int compareAndSwap(int expectedValue, int newValue) { int oldValue = value; if (oldValue == expectedValue) value = newValue; return oldValue; } public synchronized boolean compareAndSet(int expectedValue, int newValue) { return (expectedValue == compareAndSwap(expectedValue, newValue)); } } ThetypicalpatternforusingCASisfirsttoreadthevalueAfromV,derivethenewvalueBfromA,andthenuseCASto atomicallychangeVfromAtoBsolongasnootherthreadhaschangedVtoanothervalueinthemeantime.CAS addresses the problem of implementing atomic readͲmodifyͲwrite sequences without locking, because it can detect interferencefromotherthreads. 15.2.2.ANonǦblockingCounter CasCounterinListing15.2implementsathreadͲsafecounterusingCAS.Theincrementoperationfollowsthecanonical formͲfetchtheoldvalue,transformittothenewvalue(addingone),anduseCAStosetthenewvalue.IftheCASfails, theoperationisimmediatelyretried.Retryingrepeatedlyisusuallyareasonablestrategy,althoughincasesofextreme contentionitmightbedesirabletowaitorbackoffbeforeretryingtoavoidlivelock. CasCounterdoesnotblock,thoughitmayhavetoretryseveral[4]timesifotherthreadsareupdatingthecounteratthe sametime.(Inpractice,ifallyouneedisacounterorsequencegenerator,justuse AtomicIntegeror AtomicLong, whichprovideatomicincrementandotherarithmeticmethods.) [4]Theoretically,itcouldhavetoretryarbitrarilymanytimesifotherthreadskeepwinningtheCASrace;inpractice,thissortofstarvationrarely happens. Listing15.2.NonǦblockingCounterUsingCAS. @ThreadSafe public class CasCounter { private SimulatedCAS value; public int getValue() { return value.get(); } public int increment() { int v; do { v = value.get(); } while (v != value.compareAndSwap(v, v + 1)); return v + 1; } } At first glance, the CASͲbased counter looks as if it should perform worse than a lockͲbased counter; it has more operationsanda more complicated controlflow,anddependsontheseeminglycomplicatedCASoperation.Butin reality,CASͲbasedcounterssignificantlyoutperformlockͲbasedcountersifthereisevenasmallamountofcontention, andoftenevenifthereisnocontention.Thefastpathforuncontendedlockacquisitiontypicallyrequiresatleastone CASplusotherlockͲrelatedhousekeeping,somoreworkisgoingoninthebestcaseforalockͲbasedcounterthaninthe normal case for the CASͲbased counter. Since the CAS succeeds most of the time (assuming low to moderate contention),thehardwarewillcorrectlypredictthebranchimplicitinthewhileloop,minimizingtheoverheadofthe morecomplicatedcontrollogic. Thelanguagesyntaxforlockingmaybecompact,buttheworkdonebytheJVMandOStomanagelocksisnot.Locking entailstraversingarelativelycomplicatedcodepathintheJVMandmayentailOSͲlevellocking,threadsuspension,and contextswitches.Inthebestcase,lockingrequiresatleastoneCAS,sousinglocksmovestheCASoutofsightbut  198 JavaConcurrencyInPractice doesn'tsaveanyactualexecutioncost.Ontheotherhand,executingaCASfromwithintheprograminvolvesnoJVM code,systemcalls,orschedulingactivity.Whatlookslikealongercodepathattheapplicationlevelisinfactamuch shortercodepathwhenJVMandOSactivityaretakenintoaccount.TheprimarydisadvantageofCASisthatitforces the caller to deal with contention (by retrying, backing off, or giving up), whereas locks deal with contention automaticallybyblockinguntilthelockisavailable.[5] [5]Actually,thebiggestdisadvantageofCASisthedifficultyofconstructingthesurroundingalgorithmscorrectly. CAS performance varies widely across processors. On a singleͲCPU system, a CAS typically takes on the order of a handful of clock cycles, since no synchronization across processors is necessary. As of this writing, the cost of an uncontendedCASonmultipleCPUsystemsrangesfromabouttentoabout150cycles;CASperformanceisarapidly movingtargetandvariesnotonlyacrossarchitecturesbutevenacrossversionsofthesameprocessor.Competitive forceswilllikelyresultincontinuedCASperformanceimprovementoverthenextseveralyears.Agoodruleofthumbis thatthecostofthe"fastpath"foruncontendedlockacquisitionandreleaseonmostprocessorsisapproximatelytwice thecostofaCAS. 15.2.3.CASSupportintheJVM So,howdoesJavacodeconvincetheprocessortoexecuteaCASonitsbehalf?PriortoJava5.0,therewasnowaytodo thisshortofwritingnativecode.InJava5.0,lowͲlevelsupportwasaddedtoexposeCASoperationsonint,long,and objectreferences,andtheJVMcompilestheseintothemostefficientmeansprovidedbytheunderlyinghardware.On platformssupportingCAS,theruntimeinlinesthemintotheappropriatemachineinstruction(s);intheworstcase,ifa CASͲlikeinstructionisnotavailabletheJVMusesaspinlock.ThislowͲlevelJVMsupportisusedbytheatomicvariable classes (AtomicXXX in java.util.concurrent. atomic) to provide an efficient CAS operation on numeric and reference types; these atomic variable classes are used, directly or indirectly, to implement most of the classes in java.util.concurrent. 15.3.AtomicVariableClasses AtomicvariablesarefinerͲgrainedandlighterͲweightthanlocks,andarecriticalforimplementinghighͲperformance concurrentcodeonmultiprocessorsystems.Atomicvariableslimitthescopeofcontentiontoasinglevariable;thisisas fineͲgrainedasyoucanget(assumingyouralgorithmcanevenbeimplementedusingsuchfinegranularity).Thefast (uncontended)pathforupdatinganatomicvariableisnoslowerthanthefastpathforacquiringalock,andusually faster; the slow path is definitely faster than the slow path for locks because it does not involve suspending and reschedulingthreads.Withalgorithmsbasedonatomicvariablesinsteadoflocks,threadsaremorelikelytobeableto proceedwithoutdelayandhaveaneasiertimerecoveringiftheydoexperiencecontention. TheatomicvariableclassesprovideageneralizationofvolatilevariablestosupportatomicconditionalreadͲmodifyͲ writeoperations.AtomicIntegerrepresentsanintvalue,andprovidesgetandsetmethodswiththesamememory semanticsasreadsandwritestoavolatileint.ItalsoprovidesanatomiccompareAndSetmethod(whichifsuccessful hasthememoryeffectsofbothreadingandwritingavolatilevariable)and,forconvenience,atomicadd,increment, anddecrementmethods.AtomicIntegerbearsasuperficialresemblancetoanextendedCounterclass,butoffersfar greaterscalabilityundercontentionbecauseitcandirectlyexploitunderlyinghardwaresupportforconcurrency. There are twelve atomic variable classes, divided into four groups: scalars, field updaters, arrays, and compound variables.Themostcommonlyusedatomicvariablesarethescalars:AtomicInteger,AtomicLong,AtomicBoolean,and AtomicReference. All support CAS; the Integer and Long versions support arithmetic as well. (To simulate atomic variablesofotherprimitivetypes,youcancast shortor bytevaluestoandfrom int,anduse floatToIntBitsor doubleToLongBitsforfloatingͲpointnumbers.) The atomic array classes (available in Integer, Long, and Reference versions) are arrays whose elements can be updatedatomically.Theatomicarrayclassesprovidevolatileaccesssemanticstotheelementsofthearray,afeature notavailableforordinaryarraysͲa volatilearrayhasvolatilesemanticsonlyforthearrayreference,notforits elements.(TheothertypesofatomicvariablesarediscussedinSections15.4.3and15.4.4.) WhiletheatomicscalarclassesextendNumber,theydonotextendtheprimitivewrapperclassessuchasIntegeror Long. In fact, they cannot: the primitive wrapper classes are immutable whereas the atomic variable classes are mutable. The atomic variable classes also do not redefine hashCode or equals; each instance is distinct. Like most mutableobjects,theyarenotgoodcandidatesforkeysinhashͲbasedcollections.  1997BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization 15.3.1.Atomicsas"BetterVolatiles" InSection3.4.2,weusedavolatilereferencetoanimmutableobjecttoupdatemultiplestatevariablesatomically. ThatexamplereliedoncheckͲthenͲact,butinthatparticularcasetheracewasharmlessbecausewedidnotcareifwe occasionally lost an update. In most other situations, such a checkͲthenͲact would not be harmless and could compromisedataintegrity.Forexample,NumberRangeonpage67couldnotbeimplementedsafelywithavolatile referencetoanimmutableholderobjectfortheupperandlowerbounds,norwithusingatomicintegerstostorethe bounds.Becauseaninvariantconstrainsthetwonumbersandtheycannotbeupdatedsimultaneouslywhilepreserving theinvariant,anumberrangeclassusingvolatilereferencesormultipleatomicintegerswillhaveunsafecheckͲthenͲ actsequences. WecancombinethetechniquefromOneValueCachewithatomicreferencestoclosetheraceconditionbyatomically updatingthereferencetoanimmutableobjectholdingthelowerandupperbounds.CasNumberRangeinListing15.3 usesanAtomicReferencetoanIntPairtoholdthestate;byusingcompareAndSetitcanupdatetheupperorlower boundwithouttheraceconditionsofNumberRange. Listing15.3.PreservingMultivariableInvariantsUsingCAS. public class CasNumberRange { @Immutable private static class IntPair { final int lower; // Invariant: lower <= upper final int upper; ... } private final AtomicReference values = new AtomicReference(new IntPair(0, 0)); public int getLower() { return values.get().lower; } public int getUpper() { return values.get().upper; } public void setLower(int i) { while (true) { IntPair oldv = values.get(); if (i > oldv.upper) throw new IllegalArgumentException( "Can't set lower to " + i + " > upper"); IntPair newv = new IntPair(i, oldv.upper); if (values.compareAndSet(oldv, newv)) return; } } // similarly for setUpper } 15.3.2.PerformanceComparison:LocksVersusAtomicVariables To demonstrate the differences in scalability between locks and atomic variables, we constructed a benchmark comparing several implementations of a pseudorandom number generator (PRNG). In a PRNG, the next "random" numberisadeterministicfunctionofthepreviousnumber,soaPRNGmustrememberthepreviousnumberaspartof itsstate. Listings15.4and15.5showtwoimplementationsofathreadͲsafePRNG,oneusingReentrantLockandtheotherusing AtomicInteger.Thetestdriverinvokeseachrepeatedly;eachiterationgeneratesarandomnumber(whichfetchesand modifiesthesharedseedstate)andalsoperformsanumberof"busyͲwork"iterationsthatoperatestrictlyonthreadͲ localdata.Thissimulatestypicaloperationsthatincludesomeportionofoperatingonsharedstateandsomeportionof operatingonthreadͲlocalstate. Figures15.1and15.2showthroughputwithlowandmoderatelevelsofsimulatedworkineachiteration.Withalow levelofthreadͲlocalcomputation,thelockoratomicvariableexperiencesheavycontention;withmorethreadͲlocal computation,thelockoratomicvariableexperienceslesscontentionsinceitisaccessedlessoftenbyeachthread.  200 JavaConcurrencyInPractice Figure15.1.LockandAtomicIntegerPerformanceUnderHighContention.   Figure15.2.LockandAtomicIntegerPerformanceUnderModerateContention.  Listing15.4.RandomNumberGeneratorUsingReentrantLock. @ThreadSafe public class ReentrantLockPseudoRandom extends PseudoRandom { private final Lock lock = new ReentrantLock(false); private int seed; ReentrantLockPseudoRandom(int seed) { this.seed = seed; } public int nextInt(int n) { lock.lock(); try { int s = seed; seed = calculateNext(s); int remainder = s % n; return remainder > 0 ? remainder : remainder + n; } finally { lock.unlock(); } } }   2017BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization Listing15.5.RandomNumberGeneratorUsingAtomicInteger. @ThreadSafe public class AtomicPseudoRandom extends PseudoRandom { private AtomicInteger seed; AtomicPseudoRandom(int seed) { this.seed = new AtomicInteger(seed); } public int nextInt(int n) { while (true) { int s = seed.get(); int nextSeed = calculateNext(s); if (seed.compareAndSet(s, nextSeed)) { int remainder = s % n; return remainder > 0 ? remainder : remainder + n; } } } } Asthesegraphsshow,athighcontentionlevels lockingtends tooutperformatomicvariables,butatmorerealistic contentionlevelsatomicvariablesoutperformlocks.[6]Thisisbecausealockreactstocontentionbysuspendingthreads, reducingCPUusageandsynchronizationtrafficonthesharedmemorybus.(Thisissimilartohowblockingproducersin aproducerͲconsumerdesignreducestheloadonconsumersandtherebyletsthemcatchup.)Ontheotherhand,with atomic variables, contention management is pushed back to the calling class. Like most CASͲbased algorithms, AtomicPseudoRandomreactstocontentionbytryingagainimmediately,whichisusuallytherightapproachbutina highͲcontentionenvironmentjustcreatesmorecontention. [6]Thesameholdstrueinotherdomains:trafficlightsprovidebetterthroughputforhightrafficbutrotariesprovidebetterthroughputforlow traffic;thecontentionschemeusedbyEthernetnetworksperformsbetteratlowtrafficlevels,butthetokenͲpassingschemeusedbytokenring networksdoesbetterwithheavytraffic. BeforewecondemnAtomicPseudoRandomaspoorlywrittenoratomicvariablesasapoorchoicecomparedtolocks,we should realize that the level of contention in Figure 15.1 is unrealistically high: no real program does nothing but contendforalockoratomicvariable.Inpractice,atomicstendtoscalebetterthanlocksbecauseatomicsdealmore effectivelywithtypicalcontentionlevels. The performance reversal between locks and atomics at differing levels of contention illustrates the strengths and weaknessesofeach.Withlowtomoderatecontention,atomicsofferbetterscalability;withhighcontention,locksoffer bettercontentionavoidance.(CASͲbasedalgorithmsalsooutperformlockͲbasedonesonsingleͲCPUsystems,sincea CASalwayssucceedsonasingleͲCPUsystemexceptintheunlikelycasethatathreadispreemptedinthemiddleofthe readͲmodifyͲwriteoperation.) Figures15.1and15.2includeathirdcurve;animplementationofPseudoRandomthatusesaThreadLocalforthePRNG state.Thisimplementationapproachchangesthebehavioroftheclasseachthreadseesitsownprivatesequenceof pseudorandomnumbers,insteadofallthreadssharingonesequencebutillustratesthatitisoftencheapertonotshare state at all if it can be avoided. We can improve scalability by dealing more effectively with contention, but true scalabilityisachievedonlybyeliminatingcontentionentirely. 15.4.NonǦblockingAlgorithms LockͲbasedalgorithmsareatriskforanumberoflivenessfailures.Ifathreadholdingalockisdelayedduetoblocking I/O,pagefault,orotherdelay,itispossiblethatnothreadwillmakeprogress.AnalgorithmiscallednonͲblockingif failureorsuspensionofanythreadcannotcausefailureorsuspensionofanotherthread;analgorithmiscalledlockͲfree if,ateachstep,somethreadcanmakeprogress.AlgorithmsthatuseCASexclusivelyforcoordinationbetweenthreads can,ifconstructedcorrectly,bebothnonͲblockingandlockͲfree.AnuncontendedCASalwayssucceeds,andifmultiple threadscontendforaCAS,onealwayswinsandthereforemakesprogress.NonͲblockingalgorithmsarealsoimmuneto deadlockorpriorityinversion(thoughtheycanexhibitstarvationorlivelockbecausetheycaninvolverepeatedretries). We've seen one nonͲblocking algorithm so far: CasCounter. Good nonͲblocking algorithms are known for many commondatastructures,includingstacks,queues,priorityqueues,andhashtablesͲthoughdesigningnewonesisa taskbestlefttoexperts. 15.4.1.ANonǦblockingStack NonͲblockingalgorithmsareconsiderablymorecomplicatedthantheirlockͲbasedequivalents.ThekeytocreatingnonͲ blockingalgorithmsisfiguringouthowtolimitthescopeofatomicchangestoasinglevariablewhilemaintainingdata  202 JavaConcurrencyInPractice consistency. In linked collection classes such as queues, you can sometimes get away with expressing state transformations as changes to individual links and using an AtomicReference to represent each link that must be updatedatomically. Stacks are the simplest linked data structure: each element refers to only one other element and each element is referredtobyonlyoneobjectreference.ConcurrentStackinListing15.6showshowtoconstructastackusingatomic references.ThestackisalinkedlistofNodeelements,rootedattop,eachofwhichcontainsavalueandalinktothe nextelement.Thepushmethodpreparesanewlinknodewhosenextfieldreferstothecurrenttopofthestack,and thenusesCAStotrytoinstallitonthetopofthestack.Ifthesamenodeisstillonthetopofthestackaswhenwe started,theCASsucceeds;ifthetopnodehaschanged(becauseanotherthreadhasaddedorremovedelementssince westarted),theCASfailsandpushupdatesthenewnodebasedonthecurrentstackstateandtriesagain.Ineither case,thestackisstillinaconsistentstateaftertheCAS. CasCounter and ConcurrentStack illustrate characteristics of all nonͲblocking algorithms: some work is done speculatively and may have to be redone. In ConcurrentStack, when we construct the Node representing the new element,wearehopingthatthevalueofthenextreferencewillstillbecorrectbythetimeitisinstalledonthestack, butarepreparedtoretryintheeventofcontention. NonͲblocking algorithms like ConcurrentStack derive their thread safety from the fact that, like locking, compareAndSetprovidesbothatomicityandvisibilityguarantees.Whenathreadchangesthestateofthestack,itdoes sowithacompareAndSet,whichhasthememoryeffectsofavolatilewrite.Whenathreadexaminesthestack,itdoes sobycallinggetonthesameAtomicReference,whichhasthememoryeffectsofavolatileread.Soanychangesmade byonethreadaresafelypublishedtoanyotherthreadthatexaminesthestateofthelist.Andthelistismodifiedwitha compareAndSetthatatomicallyeitherupdatesthetopreferenceorfailsifitdetectsinterferencefromanotherthread. 15.4.2.ANonǦblockingLinkedList ThetwononͲblockingalgorithmswe'veseensofar,thecounterandthestack,illustratethebasicpatternofusingCAS toupdateavaluespeculatively,retryingiftheupdatefails.ThetricktobuildingnonͲblockingalgorithmsistolimitthe scopeofatomicchangestoasinglevariable.Withcountersthisistrivial,andwithastackitisstraightforwardenough, butformorecomplicateddatastructuressuchasqueues,hashtables,ortrees,itcangetalottrickier. Alinkedqueueismorecomplicatedthanastackbecauseitmustsupportfastaccesstoboththeheadandthetail.Todo this,itmaintainsseparateheadandtailpointers.Twopointersrefertothenodeatthetail:thenextpointerofthe currentlastelement,andthetailpointer.Toinsertanewelementsuccessfully,bothofthesepointersmustbeupdated atomically.Atfirstglance,thiscannotbedonewithatomicvariables;separateCASoperationsarerequiredtoupdate thetwopointers,andifthefirstsucceedsbutthesecondonefailsthequeueisleftinaninconsistentstate.And,evenif bothoperationssucceed,anotherthreadcouldtrytoaccessthequeuebetweenthefirstandthesecond.Buildinga nonͲblockingalgorithmforalinkedqueuerequiresaplanforboththesesituations.  2037BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization Listing15.6.NonǦblockingStackUsingTreiber'sAlgorithm(Treiber,1986). @ThreadSafe public class ConcurrentStack { AtomicReference> top = new AtomicReference>(); public void push(E item) { Node newHead = new Node(item); Node oldHead; do { oldHead = top.get(); newHead.next = oldHead; } while (!top.compareAndSet(oldHead, newHead)); } public E pop() { Node oldHead; Node newHead; do { oldHead = top.get(); if (oldHead == null) return null; newHead = oldHead.next; } while (!top.compareAndSet(oldHead, newHead)); return oldHead.item; } private static class Node { public final E item; public Node next; public Node(E item) { this.item = item; } } } Weneedseveraltrickstodevelopthisplan.Thefirstistoensurethatthedatastructureisalwaysinaconsistentstate, eveninthemiddleofanmultiͲstepupdate.Thatway,ifthreadAisinthemiddleofaupdatewhenthreadBarriveson thescene,Bcantellthatanoperationhasbeenpartiallycompletedandknowsnottotryimmediatelytoapplyitsown update.ThenBcanwait(byrepeatedlyexaminingthequeuestate)untilAfinishes,sothatthetwodon'tgetineach other'sway. Whilethistrickbyitselfwouldsufficetoletthreads"taketurns"accessingthedatastructurewithoutcorruptingit,if onethreadfailedinthemiddleofanupdate,nothreadwouldbeabletoaccessthequeueatall.Tomakethealgorithm nonͲblocking,wemustensurethatthefailureofathreaddoesnotpreventotherthreadsfrommakingprogress.Thus, thesecondtrickistomakesurethatifBarrivestofindthedatastructureinthemiddleofanupdatebyA,enough informationisalreadyembodiedinthedatastructureforBtofinishtheupdateforA.IfB"helps"AbyfinishingA's operation,BcanproceedwithitsownoperationwithoutwaitingforA.WhenAgetsaroundtofinishingitsoperation,it willfindthatBalreadydidthejobforit. LinkedQueue in Listing 15.7 shows the insertion portion of the MichaelͲScott nonͲblocking linkedͲqueue algorithm (MichaelandScott,1996),whichisusedbyConcurrentLinkedQueue.Asinmanyqueuealgorithms,anemptyqueue consistsofa"sentinel"or"dummy"node,andtheheadandtailpointersareinitializedtorefertothesentinel.Thetail pointeralwaysreferstothesentinel(ifthequeueisempty),thelastelementinthequeue,or(inthecasethatan operationisinmidͲupdate)thesecondͲtoͲlastelement.Figure15.3illustratesaqueuewithtwoelementsinthenormal, orquiescent,state. Figure15.3.QueuewithTwoElementsinQuiescentState.  Insertinganewelementinvolvesupdatingtwopointers.Thefirstlinksthenewnodetotheendofthelistbyupdating thenextpointerofthecurrentlastelement;thesecondswingsthetailpointeraroundtopointtothenewlastelement.  204 JavaConcurrencyInPractice Betweenthesetwooperations,thequeueisintheintermediatestate,showninFigure15.4.Afterthesecondupdate, thequeueisagaininthequiescentstate,showninFigure15.5. Figure15.4.QueueinIntermediateStateDuringInsertion.  Figure15.5.QueueAgaininQuiescentStateAfterInsertionisComplete.  Thekeyobservationthatenablesbothoftherequiredtricksisthatifthequeueisinthequiescentstate,thenextfield ofthelinknodepointedtobytailisnull,andifitisintheintermediatestate,tail.nextisnonͲnull.Soanythreadcan immediatelytellthestateofthequeuebyexaminingtail.next.Further,ifthequeueisintheintermediatestate,itcan berestoredtothequiescentstatebyadvancingthetailpointerforwardonenode,finishingtheoperationforwhichever threadisinthemiddleofinsertinganelement.[7] [7]Forafullaccountofthecorrectnessofthisalgorithm,see(MichaelandScott,1996)or(HerlihyandShavit,2006). LinkedQueue.putfirstcheckstoseeifthequeueisintheintermediatestatebeforeattemptingtoinsertanewelement (stepA).Ifitis,thensomeotherthreadisalreadyintheprocessofinsertinganelement(betweenitsstepsCandD). Ratherthanwaitforthatthreadtofinish,thecurrentthreadhelpsitbyfinishingtheoperationforit,advancingthetail pointer(stepB).Itthenrepeatsthischeckincaseanotherthreadhasstartedinsertinganewelement,advancingthetail pointeruntilitfindsthequeueinthequiescentstatesoitcanbeginitsowninsertion. TheCASatstepC,whichlinksthenewnodeatthetailofthequeue,couldfailiftwothreadstrytoinsertanelementat thesametime.Inthatcase,noharmisdone:nochangeshavebeenmade,andthecurrentthreadcanjustreloadthe tailpointerandtryagain.OnceCsucceeds,theinsertionisconsideredtohavetakeneffect;thesecondCAS(stepD)is considered"cleanup",sinceitcanbeperformedeitherbytheinsertingthreadorbyanyotherthread.IfDfails,the insertingthreadreturnsanywayratherthanretryingtheCAS,becausenoretryisneededͲanotherthreadhasalready finishedthejobinitsstepB!Thisworksbecausebeforeanythreadtriestolinkanewnodeintothequeue,itfirst checkstoseeifthequeueneedscleaningupbycheckingiftail.nextisnonͲnull.Ifitis,itadvancesthetailpointerfirst (perhapsmultipletimes)untilthequeueisinthequiescentstate.  2057BPartIV:AdvancedTopics Ͳ 27BChapter15.AtomicVariablesandNonͲblockingSynchronization Listing15.7.InsertionintheMichaelǦScottNonǦblockingQueueAlgorithm(MichaelandScott,1996). @ThreadSafe public class LinkedQueue { private static class Node { final E item; final AtomicReference> next; public Node(E item, Node next) { this.item = item; this.next = new AtomicReference>(next); } } private final Node dummy = new Node(null, null); private final AtomicReference> head = new AtomicReference>(dummy); private final AtomicReference> tail = new AtomicReference>(dummy); public boolean put(E item) { Node newNode = new Node(item, null); while (true) { Node curTail = tail.get(); Node tailNext = curTail.next.get(); if (curTail == tail.get()) { if (tailNext != null) { // Queue in intermediate state, advance tail tail.compareAndSet(curTail, tailNext); } else { // In quiescent state, try inserting new node if (curTail.next.compareAndSet(null, newNode)) { // Insertion succeeded, try advancing tail tail.compareAndSet(curTail, newNode); return true; } } } } } } 15.4.3.AtomicFieldUpdaters Listing15.7illustratesthealgorithmusedbyConcurrentLinkedQueue,buttheactualimplementationisabitdifferent. Instead of representing each Node with an atomic reference, ConcurrentLinkedQueue uses an ordinary volatile referenceandupdatesitthroughthereflectionͲbasedAtomicReferenceFieldUpdater,asshowninListing15.8. Listing15.8.UsingAtomicFieldUpdatersinConcurrentLinkedQueue. private class Node { private final E item; private volatile Node next; public Node(E item) { this.item = item; } } private static AtomicReferenceFieldUpdater nextUpdater = AtomicReferenceFieldUpdater.newUpdater( Node.class, Node.class, "next"); The atomic field updater classes (available in Integer, Long, and Reference versions) represent a reflectionͲbased "view" of an existing volatile field so that CAS can be used on existing volatile fields. The updater classes have no constructors; to create one, you call the newUpdater factory method, specifying the class and field name. The field updaterclassesarenottiedtoaspecificinstance;onecanbeusedtoupdatethetargetfieldforanyinstanceofthe targetclass.Theatomicityguaranteesfortheupdaterclassesareweakerthanfortheregularatomicclassesbecause you cannot guarantee that the underlying fields will not be modified directlythe compareAndSet and arithmetic methodsguaranteeatomicityonlywithrespecttootherthreadsusingtheatomicfieldupdatermethods. In ConcurrentLinkedQueue, updates to the next field of a Node are applied using the compareAndSet method of nextUpdater.Thissomewhatcircuitousapproachisusedentirelyforperformancereasons.Forfrequentlyallocated, shortͲlivedobjectslikequeuelinknodes,eliminatingthecreationofanAtomicReferenceforeachNodeissignificant  206 JavaConcurrencyInPractice enoughtoreducethecostofinsertionoperations.However,innearlyallsituations,ordinaryatomicvariablesperform justfineͲinonlyafewcaseswilltheatomicfieldupdatersbeneeded.(Theatomicfieldupdatersarealsousefulwhen youwanttoperformatomicupdateswhilepreservingtheserializedformofanexistingclass.) 15.4.4.TheABAProblem TheABAproblemisananomalythatcanarisefromthenaiveuseofcompareͲandͲswapinalgorithmswherenodescan berecycled(primarilyinenvironmentswithoutgarbagecollection).ACASeffectivelyasks"IsthevalueofVstillA?",and proceedswiththeupdateifso.Inmostsituations,includingtheexamplespresentedinthischapter,thisisentirely sufficient.However,sometimeswereallywanttoask"HasthevalueofVchangedsinceIlastobservedittobeA?"For somealgorithms,changingVfromAtoBandthenbacktoAstillcountsasachangethatrequiresustoretrysome algorithmicstep. ThisABAproblemcanariseinalgorithmsthatdotheirownmemorymanagementforlinknodeobjects.Inthiscase,that theheadofaliststillreferstoapreviouslyobservednodeisnotenoughtoimplythatthecontentsofthelisthavenot changed.IfyoucannotavoidtheABAproblembylettingthegarbagecollectormanagelinknodesforyou,thereisstilla relatively simple solution: instead of updating the value of a reference, update a pair of values, a reference and a version number. Even if the value changes from A to B and back to A, the version numbers will be different. AtomicStampedReference(anditscousinAtomicMarkableReference)provideatomicconditionalupdateonapairof variables.AtomicStampedReferenceupdatesanobjectreferenceͲintegerpair,allowing"versioned"referencesthatare immune[8]totheABAproblem.Similarly,AtomicMarkableReferenceupdatesanobjectreferenceͲbooleanpairthatis usedbysomealgorithmstoletanoderemaininalistwhilebeingmarkedasdeleted.[9] [8]Inpractice,anyway;theoreticallythecountercouldwrap. [9]ManyprocessorsprovideadoubleͲwideCAS(CAS2orCASX)operationthatcanoperateonapointerͲintegerpair,whichwouldmakethis operationreasonablyefficient.AsofJava6,Atomic-StampedReferencedoesnotusedoubleͲwideCASevenonplatformsthatsupportit. (DoubleͲwideCASdiffersfromDCAS,whichoperatesontwounrelatedmemorylocations;asofthiswriting,nocurrentprocessorimplements DCAS.) Summary NonͲblockingalgorithmsmaintainthreadsafetybyusinglowͲlevelconcurrencyprimitivessuchascompareͲandͲswap insteadoflocks.TheselowͲlevelprimitivesareexposedthroughtheatomicvariableclasses,whichcanalsobeusedas "bettervolatilevariables"providingatomicupdateoperationsforintegersandobjectreferences. NonͲblockingalgorithmsaredifficulttodesignandimplement,butcanofferbetterscalabilityundertypicalconditions andgreaterresistancetolivenessfailures.ManyoftheadvancesinconcurrentperformancefromoneJVMversionto thenextcomefromtheuseofnonͲblockingalgorithms,bothwithintheJVMandintheplatformlibraries.   2077BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel Chapter16.TheJavaMemoryModel Throughoutthisbook,we'vemostlyavoidedthelowͲleveldetailsoftheJavaMemoryModel(JMM)andinsteadfocused onhigherͲleveldesignissuessuchassafepublication,specificationof,andadherencetosynchronizationpolicies.These derivetheirsafetyfromtheJMM,andyoumayfinditeasiertousethesemechanismseffectivelywhenyouunderstand whytheywork.ThischapterpullsbackthecurtaintorevealthelowͲlevelrequirementsandguaranteesoftheJava MemoryModelandthereasoningbehindsomeofthehigherͲleveldesignrulesofferedinthisbook. 16.1.WhatisaMemoryModel,andWhywouldIWantOne? SupposeonethreadassignsavaluetoaVariable: aVariable = 3; Amemorymodeladdressesthequestion"UnderwhatconditionsdoesathreadthatreadsaVariableseethevalue3?" Thismaysoundlikeadumbquestion,butintheabsenceofsynchronization,thereareanumberofreasonsathread mightnotimmediatelyͲoreverͲseetheresultsofanoperationinanotherthread.Compilersmaygenerateinstructions inadifferentorderthanthe"obvious"onesuggestedbythesourcecode,orstorevariablesinregistersinsteadofin memory;processorsmayexecuteinstructionsinparalleloroutoforder;cachesmayvarytheorderinwhichwritesto variables are committed to main memory; and values stored in processorͲlocal caches may not be visible to other processors.ThesefactorscanpreventathreadfromseeingthemostupͲtoͲdatevalueforavariableandcancause memoryactionsinotherthreadstoappeartohappenoutoforderͲifyoudon'tuseadequatesynchronization. InasingleͲthreadedenvironment,allthesetricksplayedonourprogrambytheenvironmentarehiddenfromusand havenoeffectotherthantospeedupexecution.TheJavaLanguageSpecificationrequirestheJVMtomaintainwithin threadasͲifͲserialsemantics:aslongastheprogramhasthesameresultasifitwereexecutedinprogramorderina strictly sequential environment, all these games are permissible. And that's a good thing, too, because these rearrangements are responsible for much of the improvement in computing performance in recent years. Certainly higherclockrateshavecontributedtoimprovedperformance,butsohasincreasedparallelismͲpipelinedsuperscalar executionunits,dynamicinstructionscheduling,speculativeexecution,andsophisticatedmultilevelmemorycaches.As processors have become more sophisticated, so too have compilers, rearranging instructions to facilitate optimal executionandusingsophisticatedglobalregisterͲallocationalgorithms.Andasprocessormanufacturerstransitionto multicoreprocessors,largelybecauseclockratesaregettinghardertoincreaseeconomically,hardwareparallelismwill onlyincrease. In a multithreaded environment, the illusion of sequentiality cannot be maintained without significant performance cost.Sincemostofthetimethreadswithinaconcurrentapplicationareeach"doingtheirownthing",excessiveinterͲ threadcoordinationwouldonlyslowdowntheapplicationtonorealbenefit.Itisonlywhenmultiplethreadssharedata thatitisnecessarytocoordinatetheiractivities,andtheJVMreliesontheprogramtoidentifywhenthisishappening byusingsynchronization. TheJMMspecifiestheminimalguaranteestheJVMmustmakeaboutwhenwritestovariablesbecomevisibletoother threads.Itwasdesignedtobalancetheneedforpredictabilityandeaseofprogramdevelopmentwiththerealitiesof implementinghighͲperformanceJVMsonawiderangeofpopularprocessorarchitectures.SomeaspectsoftheJMM maybedisturbingatfirstifyouarenotfamiliarwiththetricksusedbymodernprocessorsandcompilerstosqueeze extraperformanceoutofyourprogram. 16.1.1.PlatformMemoryModels InasharedͲmemorymultiprocessorarchitecture,eachprocessorhasitsowncachethatisperiodicallyreconciledwith mainmemory.Processorarchitecturesprovidevaryingdegreesofcachecoherence;someprovideminimalguarantees thatallowdifferentprocessorstoseedifferentvaluesforthesamememorylocationatvirtuallyanytime.Theoperating system, compiler, and runtime (and sometimes, the program, too) must make up the difference between what the hardwareprovidesandwhatthreadsafetyrequires. Ensuringthateveryprocessorknowswhateveryotherprocessorisdoingatalltimesisexpensive.Mostofthetimethis information is not needed, so processors relax their memoryͲcoherency guarantees to improve performance. An architecture'smemorymodeltellsprogramswhatguaranteestheycanexpectfromthememorysystem,andspecifies the special instructions required (called memory barriers or fences) to get the additional memory coordination guaranteesrequiredwhensharingdata.InordertoshieldtheJavadeveloperfromthedifferencesbetweenmemory  208 JavaConcurrencyInPractice modelsacrossarchitectures,Javaprovidesitsownmemorymodel,andtheJVMdealswiththedifferencesbetweenthe JMMandtheunderlyingplatform'smemorymodelbyinsertingmemorybarriersattheappropriateplaces. Oneconvenientmentalmodelforprogramexecutionistoimaginethatthereisasingleorderinwhichtheoperations happeninaprogram,regardlessofwhatprocessortheyexecuteon,andthateachreadofavariablewillseethelast write in the execution order to that variable by any processor. This happy, if unrealistic, model is called sequential consistency.Softwaredevelopersoftenmistakenlyassumesequentialconsistency,butnomodernmultiprocessoroffers sequentialconsistencyandtheJMMdoesnoteither.Theclassicsequentialcomputingmodel,thevonNeumannmodel, isonlyavagueapproximationofhowmodernmultiprocessorsbehave. ThebottomlineisthatmodernsharedͲmemorymultiprocessors(andcompilers)candosomesurprisingthingswhen dataissharedacrossthreads,unlessyou'vetoldthemnottothroughtheuseofmemorybarriers.Fortunately,Java programs need not specify the placement of memory barriers; they need only identify when shared state is being accessed,throughtheproperuseofsynchronization. 16.1.2.Reordering In describing race conditions and atomicity failures in Chapter 2, we used interaction diagrams depicting "unlucky timing" where the scheduler interleaved operations so as to cause incorrect results in insufficiently synchronized programs. To make matters worse, the JMM can permit actions to appear to execute in different orders from the perspective of different threads, making reasoning about ordering in the absence of synchronization even more complicated.Thevariousreasonswhyoperationsmightbedelayedorappeartoexecuteoutofordercanallbegrouped intothegeneralcategoryofreordering. PossibleReordering in Listing 16.1 illustrates how difficult it is to reason about the behavior of even the simplest concurrentprogramsunlesstheyarecorrectlysynchronized.ItisfairlyeasytoimaginehowPossibleReorderingcould print(1,0),or(0,1),or(1,1):threadAcouldruntocompletionbeforeBstarts,BcouldruntocompletionbeforeA starts,ortheiractionscouldbeinterleaved.But,strangely,PossibleReorderingcanalsoprint(0,0)!Theactionsin eachthreadhavenodataflowdependenceoneachother,andaccordinglycanbeexecutedoutoforder.(Evenifthey areexecutedinorder,thetimingbywhichcachesareflushedtomainmemorycanmakeitappear,fromtheperspective of B, that the assignments in A occurred in the opposite order.) Figure 16.1 shows a possible interleaving with reorderingthatresultsinprinting(0,0). Figure16.1.InterleavingShowingReorderinginPossibleReordering.   PossibleReorderingisatrivialprogram,anditisstillsurprisinglytrickytoenumerateitspossibleresults.Reorderingat thememorylevelcanmakeprogramsbehaveunexpectedly.Itisprohibitivelydifficulttoreasonaboutorderinginthe absence of synchronization; it is much easier to ensure that your program uses synchronization appropriately. Synchronizationinhibitsthecompiler,runtime,andhardwarefromreorderingmemoryoperationsinwaysthatwould violatethevisibilityguaranteesprovidedbytheJMM.[1] [1]Onmostpopularprocessorarchitectures,thememorymodelisstrongenoughthattheperformancecostofavolatilereadisinlinewiththatof anonvolatileread. 16.1.3.TheJavaMemoryModelin500WordsorLess TheJavaMemoryModelisspecifiedintermsofactions,whichincludereadsandwritestovariables,locksandunlocks ofmonitors,andstartingandjoiningwiththreads.TheJMMdefinesapartialordering[2]calledhappensͲbeforeonall actionswithintheprogram.ToguaranteethatthethreadexecutingactionBcanseetheresultsofactionA(whetheror notAandBoccurindifferentthreads),theremustbeahappensͲbeforerelationshipbetweenAandB.Intheabsenceof ahappensͲbeforeorderingbetweentwooperations,theJVMisfreetoreorderthemasitpleases. [2]Apartialordering isarelationonasetthatisantisymmetric,reflexive,andtransitive,butforanytwoelementsxandy,itneednotbethe casethatx yory x.Weusepartialorderingseverydaytoexpresspreferences;wemayprefersushitocheeseburgersandMozarttoMahler, butwedon'tnecessarilyhaveaclearpreferencebetweencheeseburgersandMozart.  2097BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel Listing16.1.InsufficientlySynchronizedProgramthatcanhaveSurprisingResults.Don'tDothis. public class PossibleReordering { static int x = 0, y = 0; static int a = 0, b = 0; public static void main(String[] args) throws InterruptedException { Thread one = new Thread(new Runnable() { public void run() { a = 1; x = b; } }); Thread other = new Thread(new Runnable() { public void run() { b = 1; y = a; } }); one.start(); other.start(); one.join(); other.join(); System.out.println("( "+ x + "," + y + ")"); } } Adataraceoccurswhenavariableisreadbymorethanonethread,andwrittenbyatleastonethread,butthereads andwritesarenotorderedbyhappensͲbefore.Acorrectlysynchronizedprogramisonewithnodataraces;correctly synchronizedprogramsexhibitsequentialconsistency,meaningthatallactionswithintheprogramappeartohappenin afixed,globalorder. TherulesforhappensͲbeforeare: x Programorderrule.EachactioninathreadhappensͲbeforeeveryactioninthatthread thatcomeslaterinthe programorder. x Monitorlockrule.AnunlockonamonitorlockhappensͲbeforeeverysubsequentlockonthatsamemonitorlock.[3] x Volatilevariablerule.AwritetoavolatilefieldhappensͲbeforeeverysubsequentreadofthatsamefield.[4] x Threadstartrule.AcalltoThread.startonathreadhappensͲbeforeeveryactioninthestartedthread. x Thread termination rule. Any action in a thread happensͲbefore any other thread detects that thread has terminated,eitherbysuccessfullyreturnfromThread.joinorbyThread.isAlivereturningfalse. x Interruptionrule.AthreadcallinginterruptonanotherthreadhappensͲbeforetheinterruptedthreaddetectsthe interrupt(eitherbyhavingInterruptedExceptionthrown,orinvokingisInterruptedorinterrupted). x Finalizerrule.TheendofaconstructorforanobjecthappensͲbeforethestartofthefinalizerforthatobject. x Transitivity.IfAhappensͲbeforeB,andBhappensͲbeforeC,thenAhappensͲbeforeC. [3]LocksandunlocksonexplicitLockobjectshavethesamememorysemanticsasintrinsiclocks. [4]Readsandwritesofatomicvariableshavethesamememorysemanticsasvolatilevariables. Eventhoughactionsareonlypartiallyordered,synchronizationactionsͲlockacquisitionandrelease,andreadsand writes of volatile variablesͲare totally ordered. This makes it sensible to describe happensͲbefore in terms of "subsequent"lockacquisitionsandreadsofvolatilevariables. Figure16.2illustratesthehappensͲbeforerelationwhentwothreadssynchronizeusingacommonlock.Alltheactions withinthreadAareorderedbytheprogramorderrule,asaretheactionswithinthreadB.BecauseAreleaseslockM andBsubsequentlyacquiresM,alltheactionsinAbeforereleasingthelockarethereforeorderedbeforetheactionsin Bafteracquiringthelock.Whentwothreadssynchronizeondifferentlocks,wecan'tsayanythingabouttheorderingof actionsbetweenthemͲthereisnohappensͲbeforerelationbetweentheactionsinthetwothreads.  210 JavaConcurrencyInPractice Figure16.2.IllustrationofHappensǦbeforeintheJavaMemoryModel.    16.1.4.PiggybackingonSynchronization BecauseofthestrengthofthehappensͲbeforeordering,youcansometimespiggybackonthevisibilitypropertiesofan existing synchronization. This entails combining the program order rule for happensͲbefore with one of the other orderingrules(usuallythemonitorlockorvolatilevariablerule)toorderaccessestoavariablenototherwiseguarded byalock.Thistechniqueisverysensitivetotheorderinwhichstatementsoccurandisthereforequitefragile;itisan advancedtechniquethatshouldbereservedforsqueezingthelastdropofperformanceoutofthemostperformanceͲ criticalclasseslikeReentrantLock. TheimplementationoftheprotectedAbstractQueuedSynchronizermethodsinFutureTaskillustratespiggybacking. AQSmaintainsanintegerofsynchronizerstatethatFutureTaskusestostorethetaskstate:running,completed,or cancelled.ButFutureTaskalsomaintainsadditionalvariables,suchastheresultofthecomputation.Whenonethread callssettosavetheresultandanotherthreadcallsgettoretrieveit,thetwohadbetterbeorderedbyhappensͲbefore. Thiscouldbedonebymakingthereferencetotheresultvolatile,butitispossibletoexploitexistingsynchronization toachievethesameresultatlowercost. FutureTask is carefully crafted to ensure that a successful call to tryReleaseShared always happensͲbefore a subsequent call to TRyAcquireShared; try-ReleaseShared always writes to a volatile variable that is read by TRyAcquire-Shared.Listing16.2showstheinnerSetandinnerGetmethodsthatarecalledwhentheresultissavedor retrieved;sinceinnerSetwritesresultbeforecallingreleaseShared(whichcallstryReleaseShared)andinnerGet readsresultaftercallingacquireShared(whichcallsTRyAcquireShared),theprogramorderrulecombineswiththe volatilevariableruletoensurethatthewriteofresultininnerGethappensͲbeforethereadofresultininnerGet.  2117BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel Listing16.2.InnerClassofFutureTaskIllustratingSynchronizationPiggybacking. // Inner class of FutureTask private final class Sync extends AbstractQueuedSynchronizer { private static final int RUNNING = 1, RAN = 2, CANCELLED = 4; private V result; private Exception exception; void innerSet(V v) { while (true) { int s = getState(); if (ranOrCancelled(s)) return; if (compareAndSetState(s, RAN)) break; } result = v; releaseShared(0); done(); } V innerGet() throws InterruptedException, ExecutionException { acquireSharedInterruptibly(0); if (getState() == CANCELLED) throw new CancellationException(); if (exception != null) throw new ExecutionException(exception); return result; } } Wecallthistechnique"piggybacking"becauseitusesanexistinghappensͲbeforeorderingthatwascreatedforsome other reason to ensure the visibility of object X, rather than creating a happensͲbefore ordering specifically for publishingX. PiggybackingofthesortemployedbyFutureTaskisquitefragileandshouldnotbeundertakencasually.However,in somecasespiggybackingisperfectlyreasonable,suchaswhenaclasscommitstoahappensͲbeforeorderingbetween methodsaspartofitsspecification.Forexample,safepublicationusingaBlockingQueueisaformofpiggybacking.One threadputtinganobjectonaqueueandanotherthreadsubsequentlyretrievingitconstitutessafepublicationbecause thereisguaranteedtobesufficientinternalsynchronizationina BlockingQueueimplementationtoensurethatthe enqueuehappensͲbeforethedequeue. OtherhappensͲbeforeorderingsguaranteedbytheclasslibraryinclude: x Placing an item in a threadͲsafe collection happensͲbefore another thread retrieves that item from the collection; x CountingdownonaCountDownLatchhappensͲbeforeathreadreturnsfromawaitonthatlatch; x ReleasingapermittoaSemaphorehappensͲbeforeacquiringapermitfromthatsameSemaphore; x ActionstakenbythetaskrepresentedbyaFuturehappensͲbeforeanotherthreadsuccessfullyreturnsfrom Future.get; x SubmittingaRunnableorCallabletoanExecutorhappensͲbeforethetaskbeginsexecution;and x AthreadarrivingataCyclicBarrierorExchangerhappensͲbeforetheotherthreadsarereleasedfromthat samebarrierorexchangepoint.IfCyclicBarrierusesabarrieraction,arrivingatthebarrierhappensͲbefore thebarrieraction,whichinturnhappensͲbeforethreadsarereleasedfromthebarrier. 16.2.Publication Chapter3exploredhowanobjectcouldbesafelyorimproperlypublished.Thesafepublicationtechniquesdescribed therederivetheirsafetyfromguaranteesprovidedbytheJMM;therisksofimproperpublicationareconsequencesof theabsenceofahappensͲbeforeorderingbetweenpublishingasharedobjectandaccessingitfromanotherthread. 16.2.1.UnsafePublication ThepossibilityofreorderingintheabsenceofahappensͲbeforerelationshipexplainswhypublishinganobjectwithout adequatesynchronizationcanallowanotherthreadtoseeapartiallyconstructedobject(seeSection3.5).Initializinga newobjectinvolveswritingtovariablesͲthenewobject'sfields.Similarly,publishingareferenceinvolveswritingto anothervariableͲthereferencetothenewobject.IfyoudonotensurethatpublishingthesharedreferencehappensͲ  212 JavaConcurrencyInPractice beforeanotherthreadloadsthatsharedreference,thenthewriteofthereferencetothenewobjectcanbereordered (fromtheperspectiveofthethreadconsumingtheobject)withthewritestoitsfields.Inthatcase,anotherthread couldseeanupͲtoͲdatevaluefortheobjectreferencebutoutͲofͲdatevaluesforsomeorallofthatobject'sstatea partiallyconstructedobject. Unsafepublicationcanhappenasaresultofanincorrectlazyinitialization,asshowninFigure16.3.Atfirstglance,the onlyproblemhereseemstobetheraceconditiondescribedinSection2.2.2.Undercertaincircumstances,suchaswhen allinstancesoftheResourceareidentical,youmightbewillingtooverlookthese(alongwiththeinefficiencyofpossibly creating the Resource more than once). Unfortunately, even if these defects are overlooked, UnsafeLazyInitialization is still not safe, because another thread could observe a reference to a partially constructedResource. Listing16.3.UnsafeLazyInitialization.Don'tDothis. @NotThreadSafe public class UnsafeLazyInitialization { private static Resource resource; public static Resource getInstance() { if (resource == null) resource = new Resource(); // unsafe publication return resource; } } SupposethreadAisthefirsttoinvokegetInstance.Itseesthatresourceisnull,instantiatesanewResource,and setsresourcetoreferenceit.WhenthreadBlatercallsgetInstance,itmightseethatresourcealreadyhasanonͲnull valueandjustusethealreadyconstructedResource.Thismightlookharmlessatfirst,butthereisnohappensͲbefore orderingbetweenthewritingofresourceinAandthereadingofresourceinB.Adataracehasbeenusedtopublishthe object,andthereforeBisnotguaranteedtoseethecorrectstateoftheResource. TheResourceconstructorchangesthefieldsofthefreshlyallocatedResourcefromtheirdefaultvalues(writtenbythe Objectconstructor)totheirinitialvalues.Sinceneitherthreadusedsynchronization,BcouldpossiblyseeA'sactionsin a different order than A performed them. So even though A initialized the Resource before setting resource to referenceit,BcouldseethewritetoresourceasoccurringbeforethewritestothefieldsoftheResource.Bcouldthus seeapartiallyconstructed Resourcethatmaywellbeinaninvalidstateandwhosestatemayunexpectedlychange later. Withtheexceptionofimmutableobjects,itisnotsafetouseanobjectthathasbeeninitializedbyanotherthread unlessthepublicationhappensͲbeforetheconsumingthreadusesit. 16.2.2.SafePublication ThesafeͲpublicationidiomsdescribedinChapter3ensurethatthepublishedobjectisvisibletootherthreadsbecause theyensurethepublicationhappensͲbeforetheconsumingthreadloadsareferencetothepublishedobject.IfthreadA placesXonaBlockingQueue(andnothreadsubsequentlymodifiesit)andthreadBretrievesitfromthequeue,Bis guaranteed to see X as A left it. This is because the BlockingQueue implementations have sufficient internal synchronizationtoensurethattheputhappensͲbeforethetake.Similarly,usingasharedvariableguardedbyalockor asharedvolatilevariableensuresthatreadsandwritesofthatvariableareorderedbyhappensͲbefore. ThishappensͲbeforeguaranteeisactuallyastrongerpromiseofvisibilityandorderingthanmadebysafepublication. WhenXissafelypublishedfromAtoB,thesafepublicationguaranteesvisibilityofthestateofX,butnotofthestateof othervariablesAmayhavetouched.ButifAputtingXonaqueuehappensͲbeforeBfetchesXfromthatqueue,notonly doesBseeXinthestatethatAleftit(assumingthatXhasnotbeensubsequentlymodifiedbyAoranyoneelse),butB seeseverythingAdidbeforethehandoff(again,subjecttothesamecaveat).[5] [5]TheJMMguaranteesthatBseesavalueatleastasupͲtoͲdateasthevaluethatAwrote;subsequentwritesmayormaynotbevisible. Whydidwefocussoheavilyon@GuardedByandsafepublication,whentheJMMalreadyprovidesuswiththemore powerful happensͲbefore? Thinking in terms of handing off object ownership and publication fits better into most programdesignsthanthinkingintermsofvisibilityofindividualmemorywrites.ThehappensͲbeforeorderingoperates  2137BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel atthelevelofindividualmemoryaccessesͲitisasortof"concurrencyassemblylanguage".Safepublicationoperatesat alevelclosertothatofyourprogram'sdesign. 16.2.3.SafeInitializationIdioms Itsometimesmakessensetodeferinitializationofobjectsthatareexpensivetoinitializeuntiltheyareactuallyneeded, butwehaveseenhowthemisuseoflazyinitializationcanleadtotrouble.UnsafeLazyInitializationcanbefixedby makingthegeTResourcemethodsynchronized,asshowninListing16.4.BecausethecodepaththroughgetInstance isfairlyshort(atestandapredictedbranch),ifgetInstanceisnotcalledfrequentlybymanythreads,thereislittle enoughcontentionfortheSafeLazyInitializationlockthatthisapproachoffersadequateperformance. Thetreatmentofstaticfieldswithinitializers(orfieldswhosevalueisinitializedinastaticinitializationblock[JPL2.2.1 and2.5.3])issomewhatspecialandoffersadditionalthreadͲsafetyguarantees.StaticinitializersarerunbytheJVMat classinitializationtime,afterclassloadingbutbeforetheclassisusedbyanythread.BecausetheJVMacquiresalock duringinitialization[JLS12.4.2]andthislockisacquiredbyeachthreadatleastoncetoensurethattheclasshasbeen loaded,memorywritesmadeduringstaticinitializationareautomaticallyvisibletoallthreads.Thusstaticallyinitialized objectsrequirenoexplicitsynchronizationeitherduringconstructionorwhenbeingreferenced.However,thisapplies onlytotheasͲconstructedstateͲiftheobjectismutable,synchronizationisstillrequiredbybothreadersandwritersto makesubsequentmodificationsvisibleandtoavoiddatacorruption. Listing16.4.ThreadǦsafeLazyInitialization. @ThreadSafe public class SafeLazyInitialization { private static Resource resource; public synchronized static Resource getInstance() { if (resource == null) resource = new Resource(); return resource; } } Listing16.5.EagerInitialization. @ThreadSafe public class EagerInitialization { private static Resource resource = new Resource(); public static Resource getResource() { return resource; } } Using eager initialization, shown in Listing 16.5, eliminates the synchronization cost incurred on each call to getInstanceinSafeLazyInitialization.ThistechniquecanbecombinedwiththeJVM'slazyclassloadingtocreate alazyinitializationtechniquethatdoesnotrequiresynchronizationonthecommoncodepath.Thelazyinitialization holderclassidiom[EJItem48]inListing16.6usesaclasswhoseonlypurposeistoinitializetheResource.TheJVM defersinitializingtheResourceHolderclassuntilitisactuallyused[JLS12.4.1],andbecausetheResourceisinitialized with a static initializer, no additional synchronization is needed. The first call to getresource by any thread causes ResourceHoldertobeloadedandinitialized,atwhichtimetheinitializationoftheResourcehappensthroughthestatic initializer. Listing16.6.LazyInitializationHolderClassIdiom. @ThreadSafe public class ResourceFactory { private static class ResourceHolder { public static Resource resource = new Resource(); } public static Resource getResource() { return ResourceHolder.resource ; } } 16.2.4.DoubleǦcheckedLocking NobookonconcurrencywouldbecompletewithoutadiscussionoftheinfamousdoubleͲcheckedlocking(DCL)antiͲ pattern,showninListing16.7.InveryearlyJVMs,synchronization,evenuncontendedsynchronization,hadasignificant performancecost.Asaresult,manyclever(oratleastcleverͲlooking)trickswereinventedtoreducetheimpactof synchronizationͲsomegood,somebad,andsomeugly.DCLfallsintothe"ugly"category. Again,becausetheperformanceofearlyJVMsleftsomethingtobedesired,lazyinitializationwasoftenusedtoavoid potentiallyunnecessaryexpensiveoperationsorreduceapplicationstartuptime.Aproperlywrittenlazyinitialization  214 JavaConcurrencyInPractice method requires synchronization. But at the time, synchronization wasslow and, more importantly, not completely understood:theexclusionaspectswerewellenoughunderstood,butthevisibilityaspectswerenot. DCLpurportedtoofferthebestofbothworldsͲlazyinitializationwithoutpayingthesynchronizationpenaltyonthe commoncodepath.Thewayitworkedwasfirsttocheckwhetherinitializationwasneededwithoutsynchronizing,and iftheresourcereferencewasnotnull,useit.Otherwise,synchronizeandcheckagainiftheResourceisinitialized, ensuringthatonlyonethreadactuallyinitializesthesharedResource.ThecommoncodepathͲfetchingareferenceto analreadyconstructedResource doesn'tusesynchronization.Andthat'swheretheproblemis:asdescribedinSection 16.2.1,itispossibleforathreadtoseeapartiallyconstructedResource. The real problem with DCL is the assumption that the worst thing that can happen when reading a shared object referencewithoutsynchronizationistoerroneouslyseeastalevalue(inthiscase, null);inthatcasetheDCLidiom compensatesforthisriskbytryingagainwiththelockheld.ButtheworstcaseisactuallyconsiderablyworseͲitis possibletoseeacurrentvalueofthereferencebutstalevaluesfortheobject'sstate,meaningthattheobjectcouldbe seentobeinaninvalidorincorrectstate. SubsequentchangesintheJMM(Java5.0andlater)haveenabledDCLtoworkifresourceismadevolatile,andthe performanceimpactofthisissmallsincevolatilereadsareusuallyonlyslightlymoreexpensivethannonvolatilereads. However, this is an idiom whose utility has largely passedͲthe forces that motivated it (slow uncontended synchronization, slow JVM startup) are no longer in play, making it less effective as an optimization. The lazy initializationholderidiomoffersthesamebenefitsandiseasiertounderstand. Listing16.7.DoubleǦcheckedǦlockingAntiǦpattern.Don'tDothis. @NotThreadSafe public class DoubleCheckedLocking { private static Resource resource; public static Resource getInstance() { if (resource == null) { synchronized (DoubleCheckedLocking.class) { if (resource == null) resource = new Resource(); } } return resource; } } 16.3.InitializationSafety Theguaranteeofinitializationsafetyallowsproperlyconstructedimmutableobjectstobesafelysharedacrossthreads withoutsynchronization, regardlessofhowtheyarepublishedevenif published usinga datarace. (Thismeans that UnsafeLazyInitializationisactuallysafeifResourceisimmutable.) Without initialization safety, supposedly immutable objects like String can appear to change their value if synchronization is not used by both the publishing and consuming threads. The security architecture relies on the immutabilityofString;thelackofinitializationsafetycouldcreatesecurityvulnerabilitiesthatallowmaliciouscodeto bypasssecuritychecks. Initializationsafetyguaranteesthatforproperlyconstructedobjects,allthreadswillseethecorrectvaluesoffinalfields thatweresetbytheconstructor,regardlessofhowtheobjectispublished.Further,anyvariablesthatcanbereached throughafinalfieldofaproperlyconstructedobject(suchastheelementsofafinalarrayorthecontentsofaHashMap referencedbyafinalfield)arealsoguaranteedtobevisibletootherthreads.[6] [6]Thisappliesonlytoobjectsthatarereachableonlythroughfinalfieldsoftheobjectunderconstruction. Forobjectswithfinalfields,initializationsafetyprohibitsreorderinganypartofconstructionwiththeinitialloadofa referencetothatobject.Allwritestofinalfieldsmadebytheconstructor,aswellastoanyvariablesreachablethrough thosefields,become"frozen"whentheconstructorcompletes,andanythreadthatobtainsareferencetothatobjectis guaranteedtoseeavaluethatisatleastasuptodateasthefrozenvalue.Writesthatinitializevariablesreachable throughfinalfieldsarenotreorderedwithoperationsfollowingthepostͲconstructionfreeze.  2157BPartIV:AdvancedTopics Ͳ 28BChapter16.TheJavaMemoryModel Initialization safety means that SafeStates in Listing 16.8 could be safely published even through unsafe lazy initializationorstashingareferencetoaSafeStatesinapublicstaticfieldwithnosynchronization,eventhoughituses nosynchronizationandreliesonthenonͲthreadͲsafeHashSet. Listing16.8.InitializationSafetyforImmutableObjects. @ThreadSafe public class SafeStates { private final Map states; public SafeStates() { states = new HashMap(); states.put("alaska", "AK"); states.put("alabama", "AL"); ... states.put("wyoming", "WY"); } public String getAbbreviation(String s) { return states.get(s); } } However,anumberofsmallchangestoSafeStateswouldtakeawayitsthreadsafety.Ifstateswerenotfinal,orif anymethodotherthantheconstructormodifieditscontents,initializationsafetywouldnotbestrongenoughtosafely access SafeStates without synchronization. If SafeStates had other nonͲfinal fields, other threads might still see incorrect values of those fields. And allowing the object to escape during construction invalidates the initializationͲ safetyguarantee. Initializationsafetymakesvisibilityguaranteesonlyforthevaluesthatarereachablethroughfinalfieldsasofthetime theconstructorfinishes.ForvaluesreachablethroughnonͲfinalfields,orvaluesthatmaychangeafterconstruction,you mustusesynchronizationtoensurevisibility. Summary TheJavaMemoryModelspecifieswhentheactionsofonethreadonmemoryareguaranteedtobevisibletoanother. ThespecificsinvolveensuringthatoperationsareorderedbyapartialorderingcalledhappensͲbefore,whichisspecified atthelevelofindividualmemoryandsynchronizationoperations.Intheabsenceofsufficientsynchronization,some verystrangethingscanhappenwhenthreadsaccessshareddata.However,thehigherͲlevelrulesofferedinChapters2 and3,suchas@GuardedByandsafepublication,canbeusedtoensurethreadsafetywithoutresortingtothelowͲlevel detailsofhappensͲbefore.   216 JavaConcurrencyInPractice AppendixA.AnnotationsforConcurrency We've used annotations such as @GuardedBy and @ThreadSafe to show how threadͲsafety promises and synchronizationpoliciescanbedocumented.Thisappendixdocumentstheseannotations;theirsourcecode canbe downloaded from this book's website. (There are, of course, additional threadͲsafety promises and implementation detailsthatshouldbedocumentedbutthatarenotcapturedbythisminimalsetofannotations.) A.1.ClassAnnotations WeusethreeclassͲlevelannotationstodescribeaclass'sintendedthreadͲsafetypromises:@Immutable,@ThreadSafe, and @NotThreadSafe. @Immutable means, of course, that the class is immutable, and implies @ThreadSafe. @NotThreadSafeisoptionalͲifaclassisnotannotatedasthreadͲsafe,itshouldbepresumednottobethreadͲsafe,but ifyouwanttomakeitextraclear,use@NotThreadSafe. These annotations are relatively unintrusive and are beneficial to both users and maintainers. Users can see immediatelywhetheraclassisthreadͲsafe,andmaintainerscanseeimmediatelywhetherthreadͲsafetyguarantees mustbepreserved.Annotationsarealsousefultoathirdconstituency:tools.Staticcodeanalysistoolsmaybeableto verifythatthecodecomplieswiththecontractindicatedbytheannotation,suchasverifyingthataclassannotatedwith @Immutableactuallyisimmutable. A.2.FieldandMethodAnnotations TheclassͲlevelannotationsabovearepartofthepublicdocumentationfortheclass.Otheraspectsofaclass'sthreadͲ safetystrategyareentirelyformaintainersandarenotpartofitspublicdocumentation. Classesthatuselockingshoulddocumentwhichstatevariablesareguardedwithwhichlocks,andwhichlocksareused toguardthosevariables.AcommonsourceofinadvertentnonͲthreadͲsafetyiswhenathreadͲsafeclassconsistently useslockingtoguarditsstate,butislatermodifiedtoaddeithernewstatevariablesthatarenotadequatelyguardedby locking,ornewmethodsthatdonot uselockingproperlytoguardtheexistingstatevariables.Documentingwhich variablesareguardedbywhichlockscanhelppreventbothtypesofomissions. @GuardedBy(lock) documents that a field or method should be accessed only with a specific lock held. The lock argumentidentifiesthelockthatshouldbeheldwhenaccessingtheannotatedfieldormethod.Thepossiblevaluesfor lockare: x @GuardedBy("this"),meaningtheintrinsiclockonthecontainingobject(theobjectofwhichthemethodor fieldisamember); x @GuardedBy("fieldName"),meaningthelockassociatedwiththeobjectreferencedbythenamedfield,either anintrinsiclock(forfieldsthatdonotrefertoaLock)oranexplicitLock(forfieldsthatrefertoaLock); x @GuardedBy("ClassName.fieldName"),like@GuardedBy("fieldName"),butreferencingalockobjectheldina staticfieldofanotherclass; x @GuardedBy("methodName()"),meaningthelockobjectthatisreturnedbycallingthenamedmethod; x @GuardedBy("ClassName.class"),meaningtheclassliteralobjectforthenamedclass. Using@GuardedBytoidentifyeachstatevariablethatneedslockingandwhichlockguardsitcanassistinmaintenance andcodereviews,andcanhelpautomatedanalysistoolsspotpotentialthreadͲsafetyerrors.   2177BPartIV:AdvancedTopicsͲ30BBibliography Bibliography KenArnold,JamesGosling,andDavidHolmes.TheJavaProgrammingLanguage,FourthEdition.AddisonͲWesley,2005.  DavidF.Bacon,RaviB.Konuru,ChetMurthy,andMauricioJ.Serrano.ThinLocks:FeatherweightSynchronizationfor Java.InSIGPLANConferenceonProgrammingLanguageDesignandImplementation,pages258Ͳ268,1998. URLhttp://citeseer.ist.psu.edu/bacon98thin.html.  JoshuaBloch.EffectiveJavaProgrammingLanguageGuide.AddisonͲWesley,2001.  JoshuaBlochandNealGafter.JavaPuzzlers.AddisonͲWesley,2005.  HansBoehm.Destructors,Finalizers,andSynchronization.InPOPL'03:Proceedingsofthe30thACMSIGPLANͲSIGACT SymposiumonPrinciplesofProgrammingLanguages,pages262Ͳ272.ACMPress,2003. URLhttp://doi.acm.org/10.1145/604131.604153.  HansBoehm.Finalization,Threads,andtheJavaMemoryModel.JavaOnepresentation,2005. URLhttp://developers.sun.com/learning/javaoneonline/2005/coreplatform/TSͲ3281.pdf.  JosephBowbeer.TheLastWordinSwingThreads,2005. URLhttp://java.sun.com/products/jfc/tsc/articles/threads/threads3.html.  CliffClick.PerformanceMythsExposed.JavaOnepresentation,2003.  CliffClick.PerformanceMythsRevisited.JavaOnepresentation,2005. URLhttp://developers.sun.com/learning/javaoneonline/2005/coreplatform/TSͲ3268.pdf.  MartinFowler.PresentationModel,2005.URLhttp://www.martinfowler.com/eaaDev/PresentationModel.html.  ErichGamma,RichardHelm,RalphJohnson,andJohnVlissides.DesignPatterns.AddisonͲWesley,1995.  MartinGardner.ThefantasticcombinationsofJohnConway'snewsolitairegame'Life'.ScientificAmerican,October 1970.  JamesGosling,BillJoy,GuySteele,andGiladBracha.TheJavaLanguageSpecification,ThirdEdition.AddisonͲWesley, 2005.  Tim Harris and Keir Fraser. Language Support for Lightweight Transactions. In OOPSLA '03: Proceedings of the 18th Annual ACM SIGPLAN Conference on ObjectͲOriented Programming, Systems, Languages, and Applications, pages 388402.ACMPress,2003.URLhttp://doi.acm.org/10.1145/949305.949340.  TimHarris,SimonMarlow,SimonPeytonͲJones,andMauriceHerlihy.ComposableMemoryTransactions.InPPoPP'05: ProceedingsoftheTenthACMSIGPLANSymposiumonPrinciplesandPracticeofParallelProgramming,pages48Ͳ60. ACMPress,2005.URLhttp://doi.acm.org/10.1145/1065944.1065952.  MauriceHerlihy.WaitͲFreeSynchronization.ACMTransactionsonProgrammingLanguagesandSystems,13(1):124Ͳ 149,1991.URLhttp://doi.acm.org/10.1145/114005.102808.  Maurice Herlihy and Nir Shavit. Multiprocessor Synchronization and Concurrent Data Structures. MorganͲKaufman, 2006.  C.A.R.Hoare.Monitors:AnOperatingSystemStructuringConcept.CommunicationsoftheACM,17(10):549Ͳ557,1974. URLhttp://doi.acm.org/10.1145/355620.361161.  DavidHovemeyerandWilliamPugh.FindingBugsisEasy.SIGPLANNotices,39(12):92Ͳ106,2004. URLhttp://doi.acm.org/10.1145/1052883.1052895.  218 JavaConcurrencyInPractice  RamnivasLaddad.AspectJinAction.Manning,2003.  DougLea.ConcurrentProgramminginJava,SecondEdition.AddisonWesley,2000.  DougLea.JSRͲ133CookbookforCompilerWriters.URLhttp://gee.cs.oswego.edu/dl/jmm/cookbook.html.  J.D.C.Little.AproofoftheQueueingFormulaL=OW".OperationsResearch,9:383Ͳ387,1961.  JeremyManson,WilliamPugh,andSaritaV.Adve.TheJavaMemoryModel.InPOPL'05:Proceedingsofthe32ndACM SIGPLANͲSIGACTSymposiumonPrinciplesofProgrammingLanguages,pages378Ͳ391.ACMPress,2005. URLhttp://doi.acm.org/10.1145/1040305.1040336.  GeorgeMarsaglia.XorShiftRNGs.JournalofStatisticalSoftware,8(13),2003.URLhttp://www.jstatsoft.org/v08/i14.  Maged M. Michael and Michael L. Scott. Simple, Fast, and Practical NonͲBlocking and Blocking Concurrent Queue Algorithms.InSymposiumonPrinciplesofDistributedComputing,pages267Ͳ275,1996. URLhttp://citeseer.ist.psu.edu/michael96simple.html.  MarkMoirandNirShavit.ConcurrentDataStructures,InHandbookofDataStructuresandApplications,chapter47.CRC Press,2004.  WilliamPughandJeremyManson.JavaMemoryModelandThreadSpecification,2004. URLhttp://www.cs.umd.edu/~pugh/java/memoryModel/jsr133.pdf.  M.Raynal.AlgorithmsforMutualExclusion.MITPress,1986.  WilliamN.Scherer,DougLea,andMichaelL.Scott.ScalableSynchronousQueues.In11thACMSIGPLANSymposiumon PrinciplesandPracticesofParallelProgramming(PPoPP),2006.  R.K.Treiber.SystemsProgramming:CopingwithParallelism.TechnicalReportRJ5118,IBMAlmadenResearchCenter, April1986.  AndrewWellings.ConcurrentandRealͲTimeProgramminginJava.JohnWiley&Sons,2004. 

还剩233页未读

继续阅读

下载pdf到电脑,查找使用更方便

pdf的实际排版效果,会与网站的显示效果略有不同!!

需要 10 金币 [ 分享pdf获得金币 ] 0 人已下载

下载pdf

pdf贡献者

cp6p

贡献于2015-07-05

下载需要 10 金币 [金币充值 ]
亲,您也可以通过 分享原创pdf 来获得金币奖励!