We implemented enough commands for GDB to work with the stub in the previous posts. It allows GDB to attach, step executes, examine registers and memory, set breakpoints now. This post will be the last one of this series and I’d like to finish it with attaching LLDB and use not only the command line debugger but also visual debugger to test the functionalities.

6.1 LLDB

Let’s start the server as usual, but this time, instead of attaching a GDB, we’ll attach LLDB with the following commands

$ lldb
(lldb) gdb-remote localhost:2424
Process 1 stopped
* thread #1, stop reason = signal SIGTRAP
frame #0: 0xffffffffffffffff
Target 0: (No executable module.) stopped.
(lldb) target list
Current targets:
* target #0: <none> ( platform=host, pid=17, state=stopped )
(lldb) register read
error: invalid frame
(lldb) dis --arch mips error: Cannot disassemble around the current function without a selected frame.

It is successfully attached to the server stub, but doesn’t look like working perfectly. It won’t read the register values nor the memory. There must be commands that we are missing. Let’s take a look at the server log.

There are many commands that we have never seen before.

  • QThreadSuffixSupported
  • QListThreadsInStopReply
  • vCont?
  • qVAttachOrWaitSupported
  • QEnableErrorStrings
  • qHostInfo
  • qProcessInfo
  • qRegisterInfo0
  • p0
  • qStructuredDataPlugins
  • qShlibInfoAddr
  • jThreadsInfo
  • jThreadExtendedInfo

A lot of these commands are LLDB extensions listed on the spec: https://github.com/llvm/llvm-project/blob/master/lldb/docs/lldb-gdb-remote.txt

We need to support some of them to have LLDB working. The first command to support is ‘qHostInfo’ command. This is a simple command that queries the Stub’s environment such as cpu type. Supporting this command is straight forward; we just need to return a string consists of the host information.

// gdb-server-stub.js
} else if (m = packet.match(/^qHostInfo/)) {
reply = this.handler.handleHostInfo();
}
// r3000.js
handleHostInfo() {
trace('hostInfo');
// triple:mipsel-unknown-linux-gnu
return ok('triple:6d697073656c2d756e6b6e6f776e2d6c696e75782d676e75;endian:little;ptrsize:4;');
}

Note that the triple needs to be encoded as hex of the ASCII text. This is not described anywhere in the spec and confused me for some time. Let’s try attaching GDB again.

(lldb) gdb-remote localhost:2424
Process 17 stopped
* thread #1, stop reason = signal SIGTRAP
frame #0: 0xffffffffffffffff
Target 0: (No executable module.) stopped.
(lldb) target list Current targets:
* target #0: <none> ( arch=mipsel-unknown-linux-gnu, platform=remote-linux, pid=17, state=stopped )
(lldb) dis -s 0xbfc00000 0xbfc00000: addi $8, $zero, 0x3e8
0xbfc00004: addi $9, $zero, 0x7d0
0xbfc00008: add $10, $8, $9
0xbfc0000c: nop
0xbfc00010: nop
0xbfc00014: nop
0xbfc00018: nop
0xbfc0001c: nop
(lldb) register read error: invalid frame

It seems we are making some progress here. target list command properly shows the arch type as mipsel-unknown-linux-gnu. Of course what it’s attaching is not a linux-gnu, but it doesn’t really matter. I would assume that this is enough information for LLDB to properly work if the LLDB was built with MIPS target. But unfortunately the LLDB on my Mac doesn’t seem to have the register information and won’t let us read the register values. We need to support the next two commands that provide register information to LLDB. They are ‘qRegisterInfo’ and ‘p’ commands.

// “qRegisterInfo<hex-reg-id>”
//
// BRIEF
// Discover register information from the remote GDB server.
//
// PRIORITY TO IMPLEMENT
// High. Any target that can self describe its registers, should do so.
// This means if new registers are ever added to a remote target, they
// will get picked up automatically, and allows registers to change
// depending on the actual CPU type that is used.

https://github.com/llvm/llvm-project/blob/master/lldb/docs/lldb-gdb-remote.txt

and

‘p n’
Read the value of register n; n is in hex. See read registers packet, for a description of how the returned register value is encoded.

https://sourceware.org/gdb/current/onlinedocs/gdb/Packets.html#read-registers-packet

Note that we can also support a ‘qXfer:features:read:target.xml’ instead of ‘qRegisterInfo’, either way works.

This snippet shows how the commands are implemented.

The string describing a register is a list of pairs of attributes separated by ‘;’. Note that this text has to end with a ‘;’ too. Let’s attach LLDB and see what happens.

(lldb) gdb-remote localhost:2424                                                                                    Process 17 stopped
* thread #1, stop reason = signal SIGTRAP
frame #0: 0xbfc00000
-> 0xbfc00000: addi $8, $zero, 0x3e8
0xbfc00004: addi $9, $zero, 0x7d0
0xbfc00008: add $10, $8, $9
0xbfc0000c: nop
Target 0: (No executable module.) stopped.
(lldb) target list Current targets:
* target #0: <none> ( arch=mipsel-unknown-linux-gnu, platform=remote-linux, pid=17, state=stopped )
(lldb) register read General Purpose Registers:
r0 = 0x00000000
r1 = 0x00000000
r2 = 0x00000000
r3 = 0x00000000
r4 = 0x00000000
r5 = 0x00000000
r6 = 0x00000000
r7 = 0x00000000
r8 = 0x00000000
r9 = 0x00000000
r10 = 0x00000000
r11 = 0x00000000
r12 = 0x00000000
r13 = 0x00000000
r14 = 0x00000000
r15 = 0x00000000
r16 = 0x00000000
r17 = 0x00000000
r18 = 0x00000000
r19 = 0x00000000
r20 = 0x00000000
r21 = 0x00000000
r22 = 0x00000000
r23 = 0x00000000
r24 = 0x00000000
r25 = 0x00000000
r26 = 0x00000000
r27 = 0x00000000
r28 = 0x00000000
r29 = 0x00000000
r30 = 0x00000000
r31 = 0x00000000
sr = 0x00000000
lo = 0x00000000
hi = 0x00000000
bad = 0x00000000
cause = 0x00000000
pc = 0xbfc00000
(lldb) br set -H -a 0xbfc00100
Breakpoint 1: address = 0xbfc00100
(lldb) c
Process 17 resuming
Process 17 stopped
* thread #1, stop reason = breakpoint 1.1
frame #0: 0xbfc00100
-> 0xbfc00100: nop
0xbfc00104: nop
0xbfc00108: nop
0xbfc0010c: nop
Target 0: (No executable module.) stopped.
(lldb)

Perfecto! Step execution, breakpoint setting, register and memory inspection, everything seems working properly. Let’s try with a Visual Debugger too.

I’m using Visual Studio Code on Mac with a Code LLDB extension here. Step execution and register inspection are working, but the disassemble view isn’t. After some digging, I’ve discovered that it’s because the LLDB used by Code-LLDB extension doesn’t have MIPS disassembler included.

We can build a custom LLDB (liblldb) for Code LLDB including MIPS disassembler. Following are the commands I used to build a custom LLDB.

$ git clone https://github.com/llvm/llvm-project.git
$ cd llvm-project
$ mkdir build
$ cd build
$ cmake -G Xcode ../llvm -DLLVM_ENABLE_PROJECTS="clang;libcxx;lldb"
$ make -j8
$ cp -f bin/lldb ~/.vscode/extensions/vadimcn.vscode-lldb-1.5.3/lldb/
$ cp -f bin/liblldb.12.0.0git.dylib ~/.vscode/extensions/vadimcn.vscode-lldb-1.5.3/lldb/lib/liblldb.dylib

Build script is going to take sometime depending on your machine power. Note that this generates a debug build LLDB that has all symbols and asserts turned on. (There was another I bumped into along the way and I had to fix LLDB to make it work. Here’s the fix: https://reviews.llvm.org/D88375)

With everything in place, let’s give it one more try.

The GUI debugger is fully functional now!

6.2 Conclusion

I’ve started this project with a hope to implement a GDB Remote Protocol Stub from scratch that works with GDB and LLDB. I have learned some amount of the GDB/LLDB internals, npm, nodejs, ES6 and Jest along the way. A lot of things were new to me and very interesting.

I’m going to make this my last blog post for this series, but will continue updating the code. All code can be found here:https://github.com/nomtats/gdbserver-stub if anyone is interested.

--

--