Avoid having to deal with concurrency as much as possible. People good at concurrency avoid it like the plague it is.
WARNING: concurrency issues happen not only when speaking about shared memory and threads, but also between processes when contention on any kind of resource (like a database) is involved.
Example - when a job is scheduled to execute every minute by using cron.d in Linux and that job fetches and updates items from a queue persisted in MySQL, that job can take longer than 1 minute to execute and thus you can end up with 2 or 3 processes executing at the same time and contending on the same MySQL table.
Learn about the abstractions available and choose between them
depending on the task at hand. There is no silver bullet that can be
generally applied. The more high-level the abstraction, the less scope
it has in solving issues. But the less scope and power it has, the simpler
and more composable the model is. For example many developers in the Scala
community are overusing Akka Actors - which are great, but not when
misapplied. Like don't use an Akka Actor when a Future would do.
"Power tends to corrupt, and absolute power corrupts absolutely" -- Lord Acton
Scala's Futures and Promises are good because:
- they are inherently parallelizable by eliminating concurrency concerns
- fairly efficient, because when submitting a task to the implicit
ExecutionContext, this execution context efficiently multiplexes between few threads by default (the number of threads in the thread-pool is often directly proportional to the number of CPU cores you have) - alternative implementations can be even more simpler and efficient than Scala's standard Future, because the standard Future was designed to be general purpose
- the model is inherently simple and easy to use
Futures and Promises are bad because they signal only one value from the producer to the consumer and that's it - if you need a stream or bi-directional communications, a Future might not be the best abstraction.
Akka Actors are good because:
- they make bidirectional communications over asynchronous boundaries easy - for example WebSocket is a prime candidate for actors
- with Akka's Actors you can easily model state machines (see
context.become) - the processing of messages has a strong guarantee of non-concurrency - messages are processed one by one, so there's no need to worry about concurrency issues while in the context of an actor
- instead of implementing a half-assed in-memory queue for processing of things, you could just use an actor, since queuing of messages and acting on those messages is what they do
Akka's Actors are bad because:
- they are fairly low-level for many tasks
- it's extremely easy to model actors that keep a lot of state, ending up with a system that can't be horizontally scaled
- because of the bidirectional communications capability, it's extremely easy to end up with data flows that are so complex as to be unmanageable
- the model in general is actor A sending a message to actor B - but if you need to model a stream of events, this tight coupling between A and B is not acceptable
- actors, as used in practice with Akka, tend to be inducing uncontrolled side effects and that's error prone, the opposite of functional programming and not idiomatic Scala
Streaming abstractions such as Play's Iteratees / Akka Streams / RxJava / Reactive Streams / FS2 / Monix are good because:
- they model unidirectional communications between producers and consumers
- events flow in one direction and so you much easily transform and compose these streams
- depending on implementation, they address back-pressure concerns by default
- just as in the case of Future, because of the limitations, the model is simple to use and much more reasonable and composable than actors, with the exposed operators being awesome
Streams are bad because:
- they are only about unidirectional communications, it gets complicated if you want bidirectional communications, with actors being better at having dialogs
- because of the strong contract they come with (i.e. no concurrent notifications for example), implementing new operators and data-source can be problematic, but usage on the consumer side is kept simple because of this
Watch this presentation by Runar Bjarnason on this subject because it's awesome: Constraints Liberate, Liberties Constrain
This is in general an anti-pattern:
def add(x: Int, y: Int) = Future { x + y }If you don't see any kind of I/O in there, then that's a red flag. Shoving stuff in Futures without thinking is not going to solve your performance problems. Especially in the case of a web server, in which the requests are already paralellized and the above would get executed in response to requests, shoving purely CPU-bound in that Future constructor will make your logic slower to execute, not faster.
Also, in case you want to initialize a Future[T] with a constant,
always use Future.successful().
This includes all blocking I/O, including SQL queries. Real sample:
Future {
DB.withConnection { implicit connection =>
val query = SQL("select * from bar")
query()
}
}Blocking calls are error-prone because one has to be aware of exactly what thread-pool gets affected and given the default configuration of the backend app, this can lead to non-deterministic dead-locks. It's a bug waiting to happen in production.
Here's a simplified example demonstrating the issue for didactic purposes:
implicit val ec = ExecutionContext
.fromExecutor(Executors.newFixedThreadPool(1))
def addOne(x: Int) = Future(x + 1)
def multiply(x: Int, y: Int) = Future {
val a = addOne(x)
val b = addOne(y)
val result = for (r1 <- a; r2 <- b) yield r1 * r2
// this will dead-lock
Await.result(result, Duration.Inf)
}This sample is simplified to make the effect deterministic, but all thread-pools configured with upper bounds will sooner or later be affected by this.
Blocking calls have to be marked with a blocking call that signals
to the BlockContext a blocking operation. It's a very neat mechanism
in Scala that lets the ExecutionContext know that a blocking operation
happens, such that the ExecutionContext can decide what to do about
it, such as adding more threads to the thread-pool (which is what
Scala's ForkJoin thread-pool does).
All the code has to be reviewed and whenever a blocking call happens, this is the fix:
import scala.concurrent.blocking
// ...
blocking {
someBlockingCallHere()
}NOTE: the blocking call also serves as documentation, even if the
underlying thread-pool doesn't support BlockContext, as things that
block are totally non-obvious.
Sometimes you have to block the thread underneath - unfortunately JDBC doesn't have a non-blocking API. However, when you have a choice, never, ever block. For example, don't do this:
def fetchSomething: Future[String] = ???
// later ...
val result = Await.result(fetchSomething, 3.seconds)
result.toUpperCasePrefer keeping the context of that Future all the way:
def fetchSomething: Future[String] = ???
fetchSomething.map(_.toUpperCase)Also checkout Scala-Async to make this easier.
Related to Rule 4.4, if you're doing a lot of blocking I/O (e.g. a lot of calls to JDBC), it's better to create a second thread-pool / execution context and execute all blocking calls on that, leaving the application's thread-pool to deal with CPU-bound stuff.
So you could do initialize this second execution context like:
import java.util.concurrent.Executors
// ...
private val ioThreadPool = Executors.newCachedThreadPool(
new ThreadFactory {
private val counter = new AtomicLong(0L)
def newThread(r: Runnable) = {
val th = new Thread(r)
th.setName("eon-io-thread-" +
counter.getAndIncrement.toString)
th.setDaemon(true)
th
}
})Note that here I prefer to use an unbounded "cached thread-pool", so it doesn't have a limit. When doing blocking I/O the idea is that you've got to have enough threads that you can block. But if unbounded is too much, depending on use-case, you can later fine-tune it, the idea with this sample being that you get the ball rolling.
And then you could provide a helper, like:
def executeBlockingIO[T](cb: => T): Future[T] = {
val p = Promise[T]()
ioThreadPool.execute(new Runnable {
def run() = try {
p.success(blocking(cb))
}
catch {
case NonFatal(ex) =>
logger.error(s"Uncaught I/O exception", ex)
p.failure(ex)
}
})
p.future
}Also, don't expose this I/O thread-pool as an execution context that can be imported as an implicit in scope, because then people would start using it for CPU-bound stuff by mistake, so it's better to hide it and provide this helper.
As a general rule of software engineering on top of the JVM, absolutely all public APIs from inside your process will end up being used in a context in which multiple threads are using these APIs at the same time. It's best practice for all public APIs (all components in your Cake for example) to be designed as thread-safe from the get go, because you do not want to chase concurrency bugs.
And if a public API is not thread-safe for some reason (like the usual compromises made in software development), then state this fact in BOLD CAPITAL LETTERS.
The reason is simple - if an API is not thread-safe, then there's no way a user of that API can know about the way to synchronize on it. Example:
val list = mutable.List.empty[String]Lets say this mutable list is exposed. On what lock is a user supposed to synchronize? On the list itself? That's not enough, being a pretty useless lock to have in a larger context. Remember, locks are not composable and are very error-prone. Never leave the responsibility of synchronizing for contention on your users.
Meet Amdahl's Law. Synchronizing with locks drastically limits the parallelization possible and thus vertical scalability. Reads are embarrassingly paralellizable, so avoid at all times doing this:
def fetch = synchronized { someValue }Come up with better synchronization schemes that does not involve synchronizing reads, like atomic references or STM. If you aren't able to do that, then avoid this altogether by using proper abstractions.
4.9. MUST provide a clearly defined and documented protocol for each component or actor that communicates over async boundaries
A function signature is not enough for documenting the protocol of problematic components. Especially when talking about communications over asynchronous boundaries (between threads, between processes on the network, etc...), the protocol needs a lot of detail on what you can and cannot rely on.
As a guideline, don't shy away from writing comments and document:
- concurrency and latency concerns
- the proper ordering of calls
- everything that can go wrong
Shared writes are not parallelizable, whereas shared reads are embarrassingly parallelizable. As a metaphor, 100,000 people can watch the same soccer game on the same stadium at the same time (reads), but 100,000 people cannot all use the same bathroom (writes). In a multi-threading scenario, prefer single producer / multi consumer scenarios. This has the effect of avoiding contention and performance problems. An app does not scale vertically with multiple producers pounding on the same resource, because Amdahl's Law.
Checkout LMAX Disruptor.
This is a general design issue related to the project as a whole, but don't do this:
import scala.concurrent.ExecutionContext.Implicits.global
def doSomething: Future[String] = ???Tight coupling between the execution context and your logic is not good and that import is tight-coupling, especially since in the context of a Play Framework application you need to use a different thread-pool.
Just pass the ExecutionContext around as an implicit parameter. It's
idiomatic and acceptable this way. Also, implicit parameters should be passed in the second group of parameters to avoid confusing implicit resolution. When passed in the first group, it allows for method calls like doSomething() that won't compile but most IDEs will show them as valid.
def doSomething()(implicit ec: ExecutionContext): Future[String] = ???