Published: 2022-03-13

In the last couple years in Scala we’ve seen a surge in usage of various tooling protocols. A growing number of developers are at least familiar with the Language Server Protocol (LSP), especially if you’re a Metals user or were an early user of Dotty when it had a built-in LSP server. You’ve also more than likely been exposed to the Build Server Protocol (BSP) even if it was just in passing by seeing sbt create a .bsp/ directory in your Scala workspace. Another popular protocol is the Debug Adapter Protocol, which again you may have rubbed up against if you’re using Metals. I’ve seen a lot of questions about the how DAP works with Metals and this one actually has a few more moving parts than the others to make it all work together. So, I wanted to jot down some notes both to ensure I understand all the moving parts, to serve as a detailed explanation of sorts of how it all works together, and to hopefully help you to get the full picture as well.

Keep in mind that this will differ a bit per client. Client here may also be a bit ambiguous since we’ll be talking about Metals client extensions and DAP clients, so I’ll try to always differentiate them by saying “Metals client” or “DAP client” even though they may be the same thing.

Also keep in mind that this won’t really be a technical explanation of how things like expressions evaluation or breakpoints actually work, but more of an overview of all the moving pieces to better understand what happens between all of your tools when you trigger a run or debug.

What is DAP

If you’re familiar with the goals of the LSP, then you’re already familiar with some of the goals of DAP. Basically, don’t re-implement all the debugger functionality over and over for every new language and tool that wants to implement debugging. Taken from the DAP website:

Adding a debugger for a new language to an IDE or editor is not only a significant effort, but it is also frustrating that this effort can not be easily amortized over multiple development tools, as each tool uses different APIs for implementing the same feature. The idea behind the Debug Adapter Protocol (DAP) is to abstract the way how the debugging support of development tools communicates with debuggers or runtimes into a protocol. Since it is unrealistic to assume that existing debuggers or runtimes adopt this protocol any time soon, we rather assume that an intermediary component – a so called Debug Adapter – adapts an existing debugger or runtime to the Debug Adapter Protocol.

The Debug Adapter Protocol makes it possible to implement a generic debugger for a development tool that can communicate with different debuggers via Debug Adapters. And Debug Adapters can be re-used across multiple development tools which significantly reduces the effort to support a new debugger in different tools.

This description also brings up an important point of the actual Debug Adapter since in many languages that do have existing debugger interfaces built right into the language or platform, like the Java Debug Interface, which is what java-debug uses, which is what scala-debug-adapter uses, which is what most build servers will be using, which is what Metals connects to… you get the idea. However, we’ll get further into that down below. All that to say, the goal is that in a situation where a X editor user wants to transition to another editor, like Neovim, they can still have the same basic debugging experience as long as that new client has a DAP client implementation either built in or available as a plugin. The same can be said for the server side if a Metals users decides to use Bloop as their build server or sbt as their build server, they can share a common implementation of the server side of the protocol and not have to fully re-implement it twice. The server side example is exactly what the scala-debug-adapter is for.

Two different types of clients

I mentioned it up above, but also want to reiterate it here. Different language server clients may handle the DAP client part differently. Like in the case of VS Code, the DAP client is straight up just included in the editor. You can see an example of this in the scalameta/metals-vscode extension. Notice that the imports are coming right from vscode. This offers an extremely tight integration that is totally abstracted away for the user. There are other things that can then be built on top of that like the VS Code Test Explorer API that Metals recently added support for. Under the hood, the communication for these are still going through DAP. I like to think of these as “extensions” to DAP similar to LSP extensions that aren’t necessarily part of the protocol, but follow the same pattern and even re-use parts that are part of the protocol. These then require more work for other clients to implement, and they aren’t expected to work out of the box like other DAP features. For now, we won’t focus on any of these, and we’ll just stick to the core DAP features.

The other way this can look in a client is when your language server client doesn’t natively implement a DAP client, but has you use an extension to implement this. You can see an example of this in scalameta/nvim-metals. Notice that in the setup_dap function the first thing we actually do is require mfussenegger/nvim-dap which is a fantastic plugin that implements the client portion of the protocol for Neovim.

So, whether you’re using VS Code with a built-in DAP client integration or using Neovim and a plugin like nvim-dap, the core client functionality should be largely the same. Moving forward all of examples will assume the second setup using nvim-dap, since that’s what I’m most familiar with.

How does everything get set up

I don’t want this to necessarily be a “getting started with nvim-dap” guide, as there are already guides out there, plus the docs for nvim-dap are pretty detailed. Instead, I want to focus on how this all works together specifically when using nvim-metals. Most of this will be transferable to other Metals extensions that support DAP as well.

Let’s start with a simple piece of code:

@main def dapExample() =
  println("hello people interested in DAP")

If you have nvim-dap installed and you open a Scala project with a main method you should see code lenses appear on your main method. In nvim-metals it will look like this:

The first question we need to answer is “how did these get here?” and then “how does this actually trigger a run or debug of my code?”. Behind the scenes what actually happens is that Metals will have communicated with your build server and gotten any main methods in your build target via a buildTarget/scalaMainClasses request and cached those results. Then when the LSP request comes to Metals for the textDocument/codeLens Metals looks through the SemanticDB for the current document and looks for any main methods. If it finds them, it compares them to the cached ones that were retrieved earlier, and then creates code lenses for them with special commands attached to them.

Here are some illustrations of the above:

Example of the buildTarget/scalaMainClasses request and response with the build server

[Trace - 10:50:29 AM] Sending request 'buildTarget/scalaMainClasses - (7)'
Params: {
  "targets": [
    {
      "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/test/?idu003dSanity.test"
    },
    {
      "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?idu003dSanity"
    }
  ]
}

[Trace - 10:50:29 AM] Received response 'buildTarget/scalaMainClasses - (7)' in 6ms
Result: {
  "items": [
    {
      "target": {
        "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?idu003dSanity"
      },
      "classes": [
        {
          "class": "dapExample",
          "arguments": [],
          "jvmOptions": [],
          "environmentVariables": []
        }
      ]
    },
    {
      "target": {
        "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/test/?idu003dSanity.test"
      },
      "classes": []
    }
  ]
}

Example of what the SemanticDB for our code snippet looks like. Notice the first occurrence which is scala/main#. Once found we’d get the symbol for that occurrence and then check it against what was returned above.

Sanity/src/example/Hello.scala
------------------------------

Summary:
Schema => SemanticDB v4
Uri => Sanity/src/example/Hello.scala
Text => empty
Language => Scala
Symbols => 3 entries
Occurrences => 3 entries

Symbols:
_empty_/Hello$package. => final package object _empty_ extends Object { self: _empty_.type => +2 decls }
_empty_/Hello$package.dapExample(). => @main method dapExample(): Unit
_empty_/dapExample# => final class dapExample extends Object { self: dapExample => +2 decls }

Occurrences:
[0:1..0:5) => scala/main#
[0:10..0:20) <= _empty_/Hello$package.dapExample().
[1:2..1:9) => scala/Predef.println(+1).

Example of the code lens request and response.

[Trace - 07:22:25 PM] Received request 'textDocument/codeLens - (92)'
Params: {
  "textDocument": {
    "uri": "file:///Users/ckipp/Documents/scala-workspace/sanity/Sanity/src/example/Hello.scala"
  }
}

[Trace - 07:22:25 PM] Sending response 'textDocument/codeLens - (92)'. Processing request took 1ms
Result: [
  {
    "range": {
      "start": {
        "line": 0,
        "character": 1
      },
      "end": {
        "line": 0,
        "character": 5
      }
    },
    "command": {
      "title": "run",
      "command": "metals-run-session-start",
      "arguments": [
        {
          "targets": [
            {
              "uri": "file:/Users/ckipp/Documents/scala-workspace/sanity/Sanity/?idu003dSanity"
            }
          ],
          "dataKind": "scala-main-class",
          "data": {
            "class": "dapExample",
            "arguments": [],
            "jvmOptions": [],
            "environmentVariables": []
          }
        }
      ]
    }
  },
  {
    "range": {...},
    "command": {
      "title": "debug",
      "command": "metals-debug-session-start",
      "arguments": [...]
    }
  }
]

While the above is to generate the run and debug code lenses, more or less the same process happens for the test and test-debug lenses as well. The commands that are attached to the code lenses are LSP client commands that need to be implemented by the client. As you can probably guess, the two commands metals-run-session-start starts just a normal run and a metals-debug-session-start starts a debug session. How that’s done however differs a bit by client. Since we’re focusing on nvim-metals and nvim-dap I’ll outline a bit of what is happening behind the scenes to tie everything together.

Both of the commands will end up calling this function:

local function debug_start_command(no_debug)
  return function(cmd, _)
    dap.run({
      type = "scala",
      request = "launch",
      name = "from_lens",
      noDebug = no_debug,
      metals = cmd.arguments,
    })
  end
end

With nvim-dap there are two main concepts around configuration that are important to grasp. The first is the adapter configuration. This is a table given to nvim-dap per language (although the keys are actually arbitrary, just think of it per language) that basically tells nvim-dap if it should launch a debug adapter and if so how, or if it should connect to a running debugger and if so where. This configuration can be a table with these details or a function that takes a callback and a configuration. In the case of nvim-metals we use the latter with the callback, which will be explained further below. The second configuration that is relevant here is the debuggee configuration which is the configuration for your application you’ll be debugging. So if you’re familiar with VS Code think of this as your launch.json. nvim-dap can actually work using a launch.json, but we won’t focus on that here.

So in the above debug_start_command function the table being passed into dap.run() is your partially your debuggee configuration. The type is a reference to the adapter entry that matches this key, the request is either attach or launch indicating whether the debug-adapter should launch or attach to a debuggee, the name is a human readable name for the configuration (which we’ll revisit), and the noDebug is whether or not debug mode should be enabled. If this is true breakpoints will be ignored. Finally, the metals key isn’t part of the spec here and will actually be removed before being passed to nvim-dap when it fully becomes the debuggee configuration. However we use it to be able to forward the arguments from the code lens to the adapter configuration.

So where is the adapter configuration? nvim-metals fully handles the adapter configuration for you. The main reason for this is that before we can actually launch everything we need some information from Metals about how to connect to the debugger. In order to get this, we again utilize LSP to get this information before we actually start any DAP communication. Keep in mind we’ve already dove into quite a few things, but no DAP communication has even started yet. This is where the callback being part of the adapter configuration comes into play. When we setup the adapter configuration it looks something like this:

dap.adapters.scala = function(callback, config)
  local uri = vim.uri_from_bufnr(0)
  local arguments = {}

  if config.name == "from_lens" then
    arguments = config.metals
  else
    local metals_dap_settings = config.metals or {}

    arguments = {
      path = uri,
      runType = metals_dap_settings.runType or "run",
      args = metals_dap_settings.args,
      jvmOptions = metals_dap_settings.jvmOptions,
      env = metals_dap_settings.env,
      envFile = metals_dap_settings.envFile,
    }
  end

  execute_command({
    command = "metals.debug-adapter-start",
    arguments = arguments,
  }, function(_, _, res)
    if res then
      local port = util.split_on(res.uri, ":")[3]

      callback({
        type = "server",
        host = "127.0.0.1",
        port = port,
        enrich_