Marcell Ciszek Druzynski

Paralellism vs Concurrency

The difference between parallelism and concurrency is a common question in the world of software development. In this post, I'll explain the difference between the two concepts and how they are used in practice.

May 04, 2024

Understanding the disparity between concurrency and parallelism can often prove challenging. Many developers conflate these concepts and struggle to articulate their core distinctions. In this discussion, I aim to delineate the variance between concurrency and parallelism, establishing a straightforward mental framework, and elucidating the contexts in which each approach is applicable.

Consider a scenario in the kitchen, where we're preparing a sumptuous dinner for visiting friends, tackling the task solo. How can we optimize the cooking process? Let's explore three methods of cooking our fancy dish:

  • Blocking
  • Concurrent
  • Parallel

This analogy mirrors the construction of our programs. Let's commence with a typical solution employing a blocking approach.

Cooking with a Blocking Approach (Synchronous)

In a blocking approach, cooking the dinner becomes time-consuming. For instance, consider the process of boiling water. After initiating the boiling, we must wait idly until it completes rather than utilizing the time for other tasks. Subsequently, we might need to preheat the oven and wait until it reaches the desired temperature for cooking the turkey. This sequential waiting extends to each step of the process. Imagine if we have 12 distinct steps to complete; the meal preparation would significantly prolong.

Cooking with a Non-blocking Approach (Asynchronous)

Consider cooking the same dish, but this time employing an asynchronous method. We begin by boiling water once more, but instead of waiting for it to finish boiling, we proceed to the next step: preheating the oven for the turkey. We don't wait for the oven to reach the desired temperature; instead, we continue with the subsequent tasks. By executing each step asynchronously, we expedite the dish's completion and enhance productivity. This asynchronous approach optimizes our workflow.

Cooking with a Non-blocking Approach Using Multiple Chefs (Parallelism)

Now that we've explored synchronous and asynchronous methods, could we further enhance efficiency in the kitchen and handle multiple tasks concurrently? Consider a scenario where we suddenly need to prepare a dessert alongside the main dish. In this case, we enlist the help of a friend who specializes in desserts.

While I focus on preparing the main dish asynchronously, my friend can dedicate their efforts solely to crafting the dessert. This exemplifies parallelism, where multiple workers operate simultaneously, independent of one another, thus maximizing productivity without causing delays.

Where can we spot this patterns in our daily work as developers?

There are numerous instances where we can observe these patterns in our daily work as developers. Let's delve into a few examples to elucidate the distinction between concurrency and parallelism in practice. The first example using Node-js will demonstrate how asynchronous tasks can be executed concurrently, enhancing performance. The second example, utilizing Go, will showcase parallel code execution, leveraging go-routines and channels to run multiple functions concurrently.

Let's start with an asynchronies task using Node-js.

1import fs from "node:fs";
2
3// Simulate two asynchronous tasks
4function writeToFile(filename, data) {
5  return new Promise((resolve, reject) => {
6    fs.writeFile(filename, data, (err) => {
7      if (err) {
8        reject(err);
9      } else {
10        console.log(`Wrote "${data}" to ${filename}`);
11        resolve();
12      }
13    });
14  });
15}
16
17function readFromFile(filename) {
18  return new Promise((resolve, reject) => {
19    fs.readFile(filename, "utf8", (err, data) => {
20      if (err) {
21        reject(err);
22      } else {
23        console.log(`Read "${data}" from ${filename}`);
24        resolve(data);
25      }
26    });
27  });
28}
29
30async function runConcurrently() {
31  const writePromise = writeToFile("file1.txt", "Hello");
32  const readPromise = readFromFile("file2.txt");
33
34  await Promise.all([writePromise, readPromise]);
35  console.log("Both tasks completed");
36}
37
38runConcurrently();
1import fs from "node:fs";
2
3// Simulate two asynchronous tasks
4function writeToFile(filename, data) {
5  return new Promise((resolve, reject) => {
6    fs.writeFile(filename, data, (err) => {
7      if (err) {
8        reject(err);
9      } else {
10        console.log(`Wrote "${data}" to ${filename}`);
11        resolve();
12      }
13    });
14  });
15}
16
17function readFromFile(filename) {
18  return new Promise((resolve, reject) => {
19    fs.readFile(filename, "utf8", (err, data) => {
20      if (err) {
21        reject(err);
22      } else {
23        console.log(`Read "${data}" from ${filename}`);
24        resolve(data);
25      }
26    });
27  });
28}
29
30async function runConcurrently() {
31  const writePromise = writeToFile("file1.txt", "Hello");
32  const readPromise = readFromFile("file2.txt");
33
34  await Promise.all([writePromise, readPromise]);
35  console.log("Both tasks completed");
36}
37
38runConcurrently();

In this scenario, we have two asynchronous tasks: writing to a file and reading from a file. We employ the Promise API to encapsulate these asynchronous operations, and then utilize async/await to execute them concurrently.

The function runConcurrently() initiates both the write and read operations using Promise.all(), which waits for both promises to resolve before displaying the final message. This illustrates concurrency. Node js is single-threaded, but it can manage multiple asynchronous operations concurrently, enhancing performance and responsiveness.

Key points

  • Employing asynchronous functions and Promises to represent I/O tasks.
  • Initiating multiple asynchronous operations concurrently using Promise.all().
  • Ensuring the completion of concurrent operations before proceeding. This approach enables Node.js to optimize its single-threaded event loop by executing multiple I/O operations simultaneously, thereby preventing the main thread from being blocked.

Understanding How Node.js Handles Tasks Asynchronously (Event loop)

Imagine Node.js as a juggler at a circus, juggling multiple tasks at once. The event loop is like the circus stage where this juggling act happens.

Instead of waiting for each task to finish before moving on to the next, Node.js juggles them simultaneously. This keeps the show going smoothly without any delays.

Here's how it works:

  1. Different Queues: Think of these as baskets where different types of tasks are placed. There's one for timers (like setting a delay), one for I/O tasks (like reading a file or making a network request), and more.

  2. Continuous Loop: The event loop keeps spinning, constantly checking these baskets for tasks that are ready to be executed.

  3. Executing Tasks: When a task is ready, its corresponding action (like a function) is taken out of the basket and executed. This happens in a specific order, ensuring everything runs smoothly.

  4. Non-Stop Performance: Node.js handles lots of tasks without getting stuck. Even though it's single-threaded, it's like having many hands, thanks to its efficient handling of asynchronous tasks.

  5. Handling Heavy Loads: When things get really busy, Node.js doesn't collapse. It manages heavy loads by using special tools like Worker Threads, which help with parallel processing.

So, the event loop in Node.js is like the conductor of a well-organized orchestra, ensuring that every task gets its turn to shine without causing any interruptions. If you want to dive deeper into how it all works, check out the resources below.

Parallelism in code

Since Node-js does not support Parallel code execution we will demonstrate this example using Go instead.

Based on the search results, here is an example of how parallel code could look in Go:

Parallel Code Example in Go

Go makes it easy to write parallel code using go-routines and channels. Here's an example that demonstrates running multiple functions concurrently in parallel:

1package main
2
3import (
4    "fmt"
5    "sync"
6)
7
8func main() {
9    // Create a WaitGroup to synchronize the goroutines
10    var wg sync.WaitGroup
11
12    // Add 2 goroutines to the WaitGroup
13    wg.Add(2)
14
15    // Run the first function concurrently
16    go func() {
17        defer wg.Done() // Decrement the WaitGroup counter when the goroutine completes
18        doSomething1()
19    }()
20
21    // Run the second function concurrently
22    go func() {
23        defer wg.Done() // Decrement the WaitGroup counter when the goroutine completes
24        doSomething2()
25    }()
26
27    // Wait for both goroutines to complete
28    wg.Wait()
29
30    fmt.Println("All goroutines have completed")
31}
32
33func doSomething1() {
34    fmt.Println("Doing something 1...")
35    // Perform some computations or tasks in parallel
36}
37
38func doSomething2() {
39    fmt.Println("Doing something 2...")
40    // Perform some computations or tasks in parallel
41}
1package main
2
3import (
4    "fmt"
5    "sync"
6)
7
8func main() {
9    // Create a WaitGroup to synchronize the goroutines
10    var wg sync.WaitGroup
11
12    // Add 2 goroutines to the WaitGroup
13    wg.Add(2)
14
15    // Run the first function concurrently
16    go func() {
17        defer wg.Done() // Decrement the WaitGroup counter when the goroutine completes
18        doSomething1()
19    }()
20
21    // Run the second function concurrently
22    go func() {
23        defer wg.Done() // Decrement the WaitGroup counter when the goroutine completes
24        doSomething2()
25    }()
26
27    // Wait for both goroutines to complete
28    wg.Wait()
29
30    fmt.Println("All goroutines have completed")
31}
32
33func doSomething1() {
34    fmt.Println("Doing something 1...")
35    // Perform some computations or tasks in parallel
36}
37
38func doSomething2() {
39    fmt.Println("Doing something 2...")
40    // Perform some computations or tasks in parallel
41}

In this example, we create two go-routines using the go keyword, each running a separate function (doSomething1() and doSomething2()). The sync.WaitGroup is used to synchronise the go-routines, ensuring that the main function waits for both go-routines to complete before exiting.

When you run this program, you should see output similar to:

1Doing something 1...
2Doing something 2...
3All goroutines have completed
1Doing something 1...
2Doing something 2...
3All goroutines have completed

The order of the output may vary, as the go-routines are running in parallel and their execution order is not guaranteed.

This is a simple example, but you can extend it to run more functions concurrently, use channels to communicate between go-routines, and implement more complex parallel algorithms using libraries like Pargo.

Highlights

  1. Use go to create new go-routines and run functions concurrently.
  2. Use sync.WaitGroup to synchronize the go-routines and ensure the main function waits for them to complete.
  3. Go-routines can run in parallel on multiple CPU cores, but their execution order is not guaranteed.
  4. Go's concurrency primitives, like go-routines and channels, make it easier to write parallel code compared to traditional threading approaches.

Summary

It's essential to distinguish between parallelism and concurrency—they're intertwined but not synonymous.

Picture it as a large circle representing concurrency, within which lies a smaller circle denoting parallelism.

Concurrency involves managing multiple tasks or processes simultaneously, even if they don't execute concurrently. It's about interleaving tasks on a single processor, creating the illusion of parallelism.

Parallelism, however, is the real-time execution of multiple tasks across multiple processors or cores. By breaking down tasks into smaller sub-tasks processed simultaneously, parallelism boosts throughput and computational speed.

Concurrency handles multiple tasks concurrently, while parallelism executes tasks simultaneously. Techniques like context switching enable concurrency on a single processor, while parallelism demands multiple processors or cores.

In our kitchen metaphor, concurrency resembles one chef juggling tasks, while parallelism mirrors multiple chefs collaborating on different tasks.

Resources