The device was clearly working. LED on. App showing audio levels. But when I subscribed to the audio characteristic—the standard approach for every BLE audio device I'd ever worked with—nothing came through.

So, I opened Wireshark as usual and examined the raw packets.

The GATT traffic appeared normal. There was service discovery, characteristic reads, and control commands being exchanged. Then, I noticed something unusual: audio-sized packets were moving through a channel I didn't recognize.

CID 0x0041. That's not GATT.

GATT runs on CID 0x0004, a fixed channel for the ATT protocol. What I was looking at was a dynamic channel, an L2CAP Connection-Oriented Channel.

I'd read about this in the Bluetooth spec sometime ago and promptly forgotten about it, because I'd never seen it used in the real world.

Someone was actually doing it differently.

How Bluetooth LE Actually Works

Before I explain why this matters, let me walk you through how BLE moves data. If you've worked with BLE before, you probably interact with GATT: services, characteristics, notifications. But GATT is just the top layer of a stack, and understanding the layers below it explains why there's a better way.

The Stack

┌─────────────────────────────────────┐
│           Application               │  ← Your code
├─────────────────────────────────────┤
│      GATT (Generic Attribute)       │  ← Services & characteristics
├─────────────────────────────────────┤
│      ATT (Attribute Protocol)       │  ← Read/write/notify operations
├─────────────────────────────────────┤
│             L2CAP                   │  ← Packet framing & channels
├─────────────────────────────────────┤
│          Link Layer                 │  ← Radio packets
├─────────────────────────────────────┤
│         Physical Layer              │  ← 2.4 GHz radio
└─────────────────────────────────────┘

Physical Layer: The actual radio, broadcasting at 2.4 GHz.

Link Layer: Handles the raw radio packets. This is where Bluetooth defines how devices advertise, connect, and exchange data over the air. The original BLE spec (4.0) limited each packet to 27 bytes of payload. Bluetooth 4.2 introduced Data Length Extension (DLE), allowing up to 251 bytes.

L2CAP (Logical Link Control and Adaptation Protocol): Think of this as the postal service. It takes data from higher layers, adds a 4-byte header with length and channel ID, and hands it to the Link Layer. It can also segment large messages across multiple packets and reassemble them on the other side.

ATT (Attribute Protocol): Defines a simple database of "attributes"—small pieces of data identified by handles (like memory addresses). ATT provides operations to read, write, and get notifications about these attributes.

GATT (Generic Attribute Profile): Builds on ATT to create a hierarchical structure. Attributes are grouped into "characteristics" (a value plus metadata), which are grouped into "services" (a collection of related characteristics). This is the layer most developers interact with.

How Data Flows in GATT

When your fitness tracker sends your heart rate to your phone, here's what happens:

1. Your phone discovers the Heart Rate Service (UUID 0x180D)

2. Inside that service, it finds the Heart Rate Measurement characteristic

3. Your phone enables notifications by writing to a special descriptor (CCCD)

4. The tracker sends notifications whenever your heart rate changes

5. Each notification travels: GATT → ATT → L2CAP → Link Layer → Radio → Phone

The key point is that GATT and ATT always operate on a fixed L2CAP channel (CID 0x0004). Every service, characteristic, and notification is sent through this single channel.

This works great for small, infrequent data like heart rate or temperature readings. It gets awkward for streams.

The Problem with GATT for Streaming

Let's do the math on what happens when you push audio through GATT.

Packet Size Constraints

The Link Layer defines how much data fits in a single over-the-air packet:

That 23-byte or 247-byte figure is your ATT_MTU, the maximum size of an ATT operation.

But wait, there's more overhead. A GATT notification needs:

• 1 byte for the ATT opcode (notification = 0x1B)

• 2 bytes for the attribute handle

So your actual payload per notification is ATT_MTU minus 3:

What This Means for Audio

Say you're streaming 16-bit audio at 16 kHz mono. That's 32 KB per second of raw PCM data.

Without DLE (20-byte notifications):

• 32,000 ÷ 20 = 1,600 notifications per second

• Each notification has 7 bytes of overhead (L2CAP + ATT headers)

• You're sending 11,200 bytes of overhead per second just in headers

With DLE (244-byte notifications):

• 32,000 ÷ 244 = ~131 notifications per second

• Much more reasonable, but still constrained

Most devices use compressed audio (Opus, AAC) at lower bitrates, so the math isn't quite this brutal. But the fundamental problem remains.

The Bigger Problem: No Flow Control

Here's what really hurts: GATT notifications are fire-and-forget.

The server sends a notification. The client either receives it or doesn't. There's no acknowledgment, no backpressure, no way for the client to say "slow down, I'm busy."

If the client's Bluetooth stack gets temporarily overwhelmed due to a CPU spike, garbage collection, or another app using Bluetooth, packets can just vanish. The server keeps sending without realizing, and your audio experiences glitches.

For a heart rate notification every second, this rarely matters. For 131 audio packets per second, it's a real problem.

The Alternative: L2CAP Connection-Oriented Channels

Remember that L2CAP layer sitting below ATT? It turns out you can use it directly, bypassing GATT entirely.

L2CAP Connection-Oriented Channels (CoC) were added in Bluetooth 4.1. Instead of shoving everything through the fixed ATT channel, you open a dedicated channel for your data stream.

How It Works

Central (Phone)                      Peripheral (Device)
       │                                      │
       │══ LE Credit Based Connection ═══════►│
       │   PSM: 0x0080                        │
       │   MTU: 2048                          │
       │   MPS: 247                           │
       │   Initial Credits: 10                │
       │                                      │
       │◄═════ Connection Response ═══════════│
       │       Assigned CID: 0x0041           │
       │       Credits: 10                    │
       │                                      │
       │◄══════════ Data ════════════════════ │
       │◄══════════ Data ════════════════════ │
       │◄══════════ Data ════════════════════ │
       │                                      │
       │═══════ More Credits ═══════════════► │
       │           (flow control)             │
       │                                      │

Let me break down the terminology:

PSM (Protocol/Service Multiplexer): Like a port number in TCP. Identifies what protocol or service this channel is for. Values 0x0001–0x007F are reserved by the Bluetooth SIG. Values 0x0080–0x00FF are for custom applications.

CID (Channel Identifier): A unique ID for this specific channel on this specific connection. Dynamic channels use CIDs from 0x0040 to 0x007F.

MTU (Maximum Transmission Unit): The largest "message" (SDU: Service Data Unit) you can send. The spec allows up to 65,535 bytes, though memory constraints usually limit this to a few KB.

MPS (Maximum PDU Size): The largest single packet (PDU: Protocol Data Unit) on this channel. L2CAP will automatically segment larger SDUs into MPS-sized chunks.

Credits: Here's the magic. Each credit allows the sender to transmit one PDU. When you run out of credits, you stop sending. The receiver grants more credits when it's ready for more data.

Credit-Based Flow Control

This is the key difference from GATT.

Initial state:
  Device has 10 credits from Phone

Device sends audio packet #1 → Credits remaining: 9
Device sends audio packet #2 → Credits remaining: 8
Device sends audio packet #3 → Credits remaining: 7
...
Device sends audio packet #10 → Credits remaining: 0

Device must wait...

Phone finishes processing, sends 8 more credits
Device receives credits → Credits available: 8
Device resumes sending

If the phone's app is busy, it doesn't grant more credits. The device waits instead of flooding packets into the void. When the phone catches up, it grants credits and data flows again.

No dropped packets. No glitches from buffer overflow. The sender always knows the receiver is ready.

Side-by-Side Comparison

Let me put the two approaches next to each other:

GATT Notification (with DLE)

┌─────────────────┬────────────┬──────────────────────┐
│ L2CAP Header    │ ATT Header │ Payload              │
│ (4 bytes)       │ (3 bytes)  │ (up to 244 bytes)    │
└─────────────────┴────────────┴──────────────────────┘

Channel: Fixed (CID 0x0004)
Max payload: 244 bytes per notification
Max characteristic value: 512 bytes total
Flow control: None
Discovery required: Yes (services, characteristics, CCCD)

L2CAP CoC K-frame (with DLE)

┌─────────────────┬──────────────────────────────────┐
│ L2CAP Header    │ Payload                          │
│ (4 bytes)       │ (up to 247 bytes)                │
└─────────────────┴──────────────────────────────────┘
(First frame of SDU includes 2-byte SDU length field)

Channel: Dynamic (CID 0x0040–0x007F)  
Max payload: 247 bytes per PDU, up to 65,535 bytes per SDU
Flow control: Credit-based
Discovery required: No (just need to know the PSM)

The per-packet efficiency is similar, both are around 97% payload. The wins for CoC are:

1. Flow control prevents data loss under load

2. Larger logical units (64KB SDU vs 512-byte characteristic)

3. No GATT overhead (service discovery, handles, CCCDs)

4. Symmetric bidirectional (both sides equally efficient)

The Ecosystem Gap

When I tried to work with L2CAP CoC from my usual tools, I ran into a wall.

# bleak - the standard Python BLE library
# L2CAP CoC support: None
# GitHub issue #598: closed as wontfix

Bleak is a GATT client. It doesn't expose L2CAP directly, and the maintainers have decided that's outside scope.

The pattern repeats across cross-platform tools:

The native SDKs support it. The cross-platform libraries don't. If you're building with Flutter or React Native; which many teams choose for faster iteration, you'd need to drop into Swift and Kotlin separately.

That's not a dealbreaker for everyone, but it explains why most tutorials, Stack Overflow answers, and sample code stick to GATT.

The Documentation Gap

Search "BLE GATT tutorial" and you'll find hundreds of results. Nordic, TI, Espressif, Silicon Labs—every chip vendor publishes getting-started guides with working sample projects.

Search "BLE L2CAP CoC tutorial" and you get the Bluetooth Core Specification and a handful of sparse API references.

When I needed to understand the credit-based flow control details, I ended up reading the spec. It's not that the information doesn't exist, it's that nobody has packaged it into the kind of step-by-step guides that make GATT feel approachable.

When CoC Makes Sense

Despite the tooling gap, L2CAP CoC is worth considering for specific use cases:

Streaming where reliability matters: If dropped packets mean audible glitches or corrupted data, credit-based flow control prevents the silent failures that GATT notifications allow.

Bulk transfers: Firmware updates, file sync, log downloads. The 512-byte characteristic limit in GATT requires chunking logic. CoC can send larger SDUs natively.

Controlled ecosystems: If you build both the firmware and the app—and don't need third-party integrations—the compatibility concerns shrink. You're writing native code anyway.

Bidirectional real-time data: Control systems where both directions need equal efficiency and guaranteed delivery.

A clean architecture separates concerns:

GATT (control plane)
├── Device configuration
├── Status queries  
├── PSM advertisement
└── Standard services (Device Info, Battery)

L2CAP CoC (data plane)
├── Audio streaming
├── File transfer
└── High-frequency sensor data

Use the database for database things. Use the pipe for pipe things.

LE Audio Changes the Equation

Bluetooth LE Audio is now shipping on recent devices; AirPods Pro (2nd gen), iPhone 14 and later, Samsung Galaxy S23 series, Pixel 7 and up. The LC3 codec and isochronous channels provide a standardized path for audio streaming that's built into the spec.

For new products targeting current hardware, LE Audio is the right answer. It handles the codec, the transport, and the synchronization. You don't need to roll your own.

But LE Audio requires Bluetooth 5.2+ hardware on both ends. The installed base of older devices—phones from 2020, fitness trackers, smart home gadgets—won't get LE Audio support through software updates. That tail is long.

If you're building for the current generation, use LE Audio. If you need to support older devices, or you're working on something LE Audio doesn't cover (non-audio bulk data, custom protocols), L2CAP CoC remains the better-than-GATT option that most developers don't know exists.

Quick Reference

Key Numbers

Platform Support

L2CAP CoC Terminology

Well, not intentionally.

A few days before the Meta acquisition announcement, I ordered a Limitless Pendant. Payment processed. Shipping confirmed. Then, before it ever arrived, everything changed. The company announced geoblocking in multiple countries, major compliance changes, a halt on hardware sales, and a shutdown timeline for the app.

My pendant was still somewhere over the Atlantic, already obsolete.

For most people, that would have been the end of the story. Eat the cost. Move on. Maybe write an angry tweet.

But I work at Omi. We had been adding support for every major wearable audio device on the market. Limitless was the last holdout. Overnight, thousands of users in geoblocked countries were searching for alternatives. Many did not want their voice data feeding a new ecosystem. Many already owned hardware they trusted and did not want to throw away.

If I could crack the protocol, those users would get a new home. And my not yet arrived paperweight would get a second life.

There was just one problem.

I did not have the device.

Working Blind

What I did have was a friend in London.

He owned a Limitless Pendant, still had full access to the official app, and happened to be on a business trip. When this all started, he was in the back of an Uber, on the way to the airport, heading back to the United States.

When I asked if he could run a few scripts, he did not hesitate. He opened his laptop in the car.

Over the next 24 hours, he became my Bluetooth lab, my QA team, and my reality check. Everything I learned came through him. I would write scripts, send them over, he would run them, and I would stare at logs trying to reconstruct what was happening five thousand miles away.

I had reverse engineered enough BLE audio devices to know where to start. I wrote a small Python script using bleak to scan everything the pendant advertised: services, characteristics, properties. He ran it from the Uber.

The logs came back clean.

Service: 632de001-604c-446b-a80f-7963e950f3fb
  Characteristic: 632de002-604c-446b-a80f-7963e950f3fb
    Properties: ['write', 'write-without-response']
  Characteristic: 632de003-604c-446b-a80f-7963e950f3fb
    Properties: ['notify']

Three sequential UUIDs. Classic BLE architecture. ...02 for sending commands to the device. ...03 for receiving data back. Nothing exotic. This was workable.

I wrote another script to connect and capture every packet the pendant emitted. Between traffic lights, airport Wi-Fi, and boarding announcements, he kept running commands. After a lot of back and forth, reconnects, Bluetooth restarts, and button presses, we got stable connections.

But no audio.

The pendant just sat there. LED dark. Completely silent.

The Missing Piece

I tried everything. Different command sequences. Different timing. Different connection orders. Nothing worked.

So I stopped trying to talk to the pendant and started listening to what the official app was saying to it.

I walked my friend through enabling developer options on his Android phone. He used the pendant normally while capturing HCI logs, the raw Bluetooth traffic between phone and device. He did this several times, restarting between runs.

By then, he was at the airport.

By the time his flight boarded, I had a stack of packet captures in my inbox.

I loaded them into Wireshark and compared sessions side by side. Hex dumps blur together after a while. Then something stood out.

One packet appeared in every capture. Same structure. Same position in the handshake. Every time.

32 07 08 c1 97 c6 c2 af 33

I did not immediately recognize the format. I dumped a batch of packets into Claude and asked what it could identify.

Those look like protobuf field tags.

Of course.

I decoded it by hand.

• 32 means field 6, length delimited

• 07 means seven bytes follow

• 08 means nested field 1, varint

• c1 97 c6 c2 af 33 is a varint encoded value

The decoded value was 1765102684161.

A Unix timestamp in milliseconds. December 7, 2025.

The app was telling the pendant what time it was.

That was the missing piece.

The pendant timestamps all audio internally. Without knowing the current time, it literally cannot record. It was not broken. It was not locked down. It was waiting.

I wrote the encoder in about thirty seconds.

def encode_set_current_time(timestamp_ms: int) -> bytes:
    time_varint = bytes([0x08]) + encode_varint(timestamp_ms)
    return bytes([0x32, len(time_varint)]) + time_varint

I sent the updated script.

By then, he was taxiing for takeoff.

He ran it again after reconnecting.

Then he pressed the button.

The LED turned on.

For the first time, without the official app, the device was recording.

Cracking the Stream

Recording and getting usable audio are different problems.

Packets were flowing now, even as his plane climbed out of London. The data looked like noise. I needed to identify the codec.

I made the obvious guess first. Opus at 16 kHz. Almost everyone uses it for voice.

Sure enough, certain bytes repeated at consistent offsets: 0xb8, 0x78, 0xf8.

I pulled up RFC 6716, the Opus specification. These were TOC bytes. Table of Contents. They encode frame configuration, mono or stereo, and frame count. 0xb8 decodes to config 23, mono, single frame. Exactly what you would expect from a wearable microphone.

The pendant was encoding 20 ms Opus frames and wrapping each one in a protobuf message.

I extracted a few dozen frames, stitched them together, wrote them to an Ogg file, and hit play.

My friend’s voice came through the speakers.

Spectrogram of the first successful extraction. That's speech.

A little crackly at the transitions, but unmistakably real. Recorded on a device I had never touched, using a protocol I had learned existed only hours earlier.

Ship It

For anyone keeping score, the full stack looked like this.

┌─────────────────────────────────────────────┐
│ Application Layer (Opus audio frames)       │
├─────────────────────────────────────────────┤
│ Message Layer (Protobuf fields)             │
├─────────────────────────────────────────────┤
│ Fragment Layer (sequence and count)         │
├─────────────────────────────────────────────┤
│ BLE GATT (TX: ...02 / RX: ...03)            │
└─────────────────────────────────────────────┘

Once the Python proof of concept worked, I ported everything to Dart for our Flutter app. I pushed a build to TestFlight and sent it to a handful of Limitless users who had reached out.

It works.

My recordings show up.

I can export them.

It is real.

By the time my collaborator landed in the United States, the protocol was cracked.

We pushed it live to the App Store the next day.

Omi traffic after Limitless support went live.

No encryption was bypassed. No proprietary code was decompiled. The protocol uses standard BLE, standard Protobuf, and standard Opus. All open specifications.

This was pattern recognition, persistence, and one very patient collaborator with a device I could not touch.

My pendant finally arrived three days later.

By then, it already worked with Omi.

Screenshots and code samples have been simplified for clarity. The actual implementation is in the PR.

If you are reading this, then chances are you already know what Google Summer of Code is and how it works. You might want to check out their official site for the perfect explanation if you don't. In brief, GSoC is an open-source program powered by Google that connects organizations to students and professionals to work on their projects over the summer. This is an amazing opportunity for students as they are exposed to real-world code and also get first-hand experience working on production-level projects. And for working professionals, it is an opportunity to dive into the world of open source by making meaningful contributions to open-source projects.

GSoC is not an internship but more of a program to bring people into open-source development. You are, of course, paid decently for your work, which is an additional perk. The stipend varies from country to country and is listed on their website. There are three types of projects - small (~90 hours), medium (~175 hours), and large (~350 hours), categorized based on the time span contributors need to dedicate to that project.

The Journey

Back in January 2024, I was preparing for Summer of Bitcoin. I didn’t know that GSoC was now open to everyone (it was not restricted to just students) and I wasn’t planning to do my master's either, so I did not think about it at all. This was my last chance for Summer of Bitcoin, and I really wanted to be a part of it.

Fast forward to March, and I was trying to work on the Summer of Bitcoin assessment that had to be done as part of the proposal submission. I came across a post about GSoC, looked it up, and thought, why not give it a chance one last time? I had a few contacts who had been part of GSoC in the past. I reached out to them with a few questions and stuff. It was around March 20, and there were only about 12 days left to submit the proposal. The SoB deadline was already approaching, but the assessment wasn’t easy, so I thought of doing it later (spoiler: I did not do it at all).

I knew by now that many people would have contributed and become comfortable with the projects and the community. I did not have any advantage over them (except that maybe I had some industry experience). I was planning to apply for projects under the CCExtractor organization, so I completed their required introductory tasks like creating a Grafana dashboard for MyFitnessPal data and creating a macOS release for CCExtractor. But I still was not confident that I would make the cut, so I started looking for other organizations and came across AOSSIE.

AOSSIE had many projects, but I found one project particularly interesting because no one was able to run it correctly at all since the codebase was very old, and it was a native Android app with a Flutter module. That project was Monumento. I spent the next 2-3 days trying to migrate it to a complete Flutter app, fixing many native Android issues, and also the AR functionality. In about 3 days, I had the app fully functional and working with my Firebase. Then an organization admin gave me access to their Firebase account to set this project correctly to use their account with all the data already there.

The deadline was approaching, and I had about 3-4 days to create a proposal, get it reviewed by a mentor, and submit it. I spent the entire day going through many old proposals to understand how everyone did it and what makes a proposal stand out. The very next day, I started working on it and made my first draft. I sent it to my project mentor, and he gave some feedback. I quickly started making changes based on his feedback. His feedback was basically to propose some new features alongside the UI redesign, so I spent some time creating MVPs of the features I was planning and also almost finished the UI design in Figma.

I spent quite a lot of time in the last 2-3 days to make my proposal stand out from the crowd. I submitted it on the deadline day, waited for a month, and when the results came, I was ready to accept my rejection (like in previous years), but it was a selection. It was one of the happiest days for me.

Everyone’s Journey is Different

I've read many blogs where people share their GSoC selection stories, and most of them mention starting their preparation in December of the previous year. While that's the ideal way to begin, don't lose hope if you're starting a bit late. This was the only reason for me to write this blog and share my story. Remember, you haven't lost until the results are announced. If you quit early, you've lost before the results even come out. Believe in yourself, and you’ll succeed! :)

Some Tips

• Don't do it just for the money :)

• Your proposal determines your outcome, so make it as detailed as possible and get it reviewed early.

• It's never too early or too late to start. You can even begin just a day before the deadline (but please don't).

• Don't focus on too many projects. Limit yourself to a maximum of 2-3 projects so you can dedicate time to each.

• Quality over quantity. Focus on making meaningful contributions.

The majority of Cattleguru's audience resides in villages and rural areas in Northern India. While all major cities already have 5G and many tier 2 cities are on 4G, rural areas still lag behind. Creating an e-commerce app for these regions is quite challenging due to limited connectivity. Additionally, people often avoid updating the app because of limited internet data. This makes it difficult to make online-only apps. Furthermore, the loading time for fetching data from the backend and displaying it in the app can be significant, sometimes causing users to wait for more than 10-15 seconds to perform any operation.

Background

We needed a solution to:

• Reduce latency for all sorts of operations originating from our Apps

• Keep App Data in Sync with Remote Database

Initially, we didn't use any specific solution; the app simply fetched data from Firestore and displayed it. Given that Firestore caches data locally, we didn't consider alternatives until we observed real-world app usage. We noticed that users weren't using the app frequently; instead, they placed orders through our salesmen. This was because most of the time, they saw the app taking anywhere between 20-30 seconds to successfully place an order. Hence, they found it easier to just call the salesmen and place the order. As we expanded into more remote villages, our delivery partners couldn't effectively use the internal team app to deliver orders due to low connectivity in those remote villages. Therefore, we needed a more robust and efficient solution to improve operations.

Approaches

There are various approaches we could have taken to resolve our issues. One approach was using CRDTs (Conflict-Free Replicated Data Type). This would have made sense if our apps were write-heavy and collaborative. While we considered it initially, we discarded it because our apps are mostly read-heavy, and only the internal team app is somewhat write-heavy. You might wonder why we didn't use CRDTs for the team app since it is write-heavy. Although it is both read and write-heavy, there are other approaches with less overhead compared to CRDTs, which make more sense because we wouldn't fully utilize CRDTs even if we implemented them.

Another approach was to make our apps offline-first with sync services like Electric SQL or PowerSync. However, the problem here is that we don't use Postgres (yet), so migrating from Firestore to PostgreSQL would be a time taking task. Additionally, Electric SQL is not yet stable and has many limitations at present, while PowerSync isn't open source yet.

Another approach is to cache whatever we can and move all the time-consuming operations to the cloud (backend). We chose this approach. After quite some research, we decided to follow a three-layered caching strategy coupled with feature flags and message queues for our systems.

The Chosen Approach

If you want to reduce the latency of fetching frequently accessed and infrequently changed data, caching is one of the best solutions with very little overhead. That's exactly what we did by adding caching at various levels. We have a client-side database, an in-memory (non-persistent) database at the server level, and an adjacent Redis instance. A call is made to Firestore only if the data does not exist at the previous three levels.

Our SKUs don't change frequently, and if there is a change, it's mostly in the price. So, it made sense for us to cache the product data on the client side for the long term. Whenever the app opens, it makes one API call to fetch the last cache update time and feature flags update time and compares it locally. To minimize the amount of data sent to the client, we only return the cache keys and feature flag keys that have changed. This might not seem like much, but it helps save a few bytes of data.

When it comes to our internal team app, the order data changes every time a new order is received. Sometimes we don't receive orders for hours. Therefore, it doesn't make sense to have a fixed expiry for the cache while updating it at specific intervals, especially since we don't receive orders between 1 AM and 6 AM at all. Instead, we decided to use a cache with no expiry. Rather than updating both the in-memory cache and Redis with every new order, we opted to use Cron Jobs to update them at regular intervals. The Cron Jobs are scheduled to run only during the working hours of our teams. For those curious, we use Upstash Redis and QStash for this purpose.

To minimize the amount of data written by every write operation of the team app, we redesigned the feature flows to reduce as much data as possible while ensuring enough data is captured. Since we already know who our delivery partners are and the customers who placed the orders, we eliminated all unnecessary fields being transferred to the backend, using only one field as a strong identifier of the user.

One very common problem with having caches at multiple levels is keeping all of them in sync with the main database. This is a big issue if the data changes frequently. But given the data that is cached locally on the client side doesn't change that frequently, it isn't that big of an issue for us. We use flags to keep track of what data was changed and when it changed. It helps us in keeping the data between the client and the backend in sync.

The table below shows how much the response time has improved, thanks to the approaches we implemented.

Note: You will have to be on the "Pay as you go" plan of Oracle Cloud Infrastructure (OCI) for you to be able to create an Ampere VM (with upto 24GB RAM and 4OCPUs for free).

VM Setup

Spin up an Ampere VM with at least 4GB of RAM (I opted for 8GB) through OCI's instance creation UI. Make sure to select Ubuntu 20.04 or later as the base image.

Important: Make sure you download the private as well as the public keys (you won't be able to SSH into the VM later on if you don't have the private key).

The newly created instance should be up and running in a minute or two.

SSH into VM

Open a new terminal on your local machine and change the permissions for the private key by running the following command:

chmod 400 /path/to/key/key_name.key

After changing the permissions, SSH into the VM by running the following command:

ssh username@<vm-ip-address> -i /path/to/key/key_name.key

You can find the username and public IP of your VM in the VM Instance details page on OCI.

Installing Core Lightning (Approaches)

There are three ways to install Core Lightning

1. Installing pre-compiled binaries

2. Using Docker

3. Building Binaries from source

Since we are on an Arm based VM, we won't be able to follow the first approach because Core Lightning only provides pre-compiled binaries for AMD-based Fedora and Ubuntu distributions, which can be found here.

We will be following the third approach, which involves compiling the binary from source.

Installing Dependencies

The following commands will download all the required dependencies:

sudo apt-get update
sudo apt-get install -y \
  autoconf automake build-essential git libtool libsqlite3-dev \
  python3 python3-pip net-tools zlib1g-dev libsodium-dev gettext
pip3 install --upgrade pip
pip3 install --user poetry

Installing Bitcoin

We will not be running our own Bitcoin Core node due to storage constraints (it requires around 400GB of storage). Instead, we will install it minimally:

sudo apt-get install snapd
sudo snap install bitcoin-core
# Snap does some weird things with binary names; you'll
# want to add a link to them so everything works as expected
sudo ln -s /snap/bitcoin-core/current/bin/bitcoin{d,-cli} /usr/local/bin/

Setting up Core Lightning

The process isn't really different from what is in the documentation. So I have copied and pasted the required commands to be followed.

git clone https://github.com/ElementsProject/lightning.git
cd lightning
git checkout v23.11.2
sudo apt-get install -y valgrind libpq-dev shellcheck cppcheck \
  libsecp256k1-dev jq lowdown

Building Core Lightning

pip3 install --upgrade pip
pip3 install mako
pip3 install -r plugins/clnrest/requirements.txt
pip3 install grpcio-tools
./configure
make
sudo make install

By this point, you should have the lightning and Bitcoin binaries set up correctly. If you attempt to run the command lightningd, you'll probably get some error stating that there's no Bitcoin node running. To fix this, we have to connect to a Bitcoin node. For this, we can either run our own Bitcoin node (which we already have installed) or use a third-party plugin. Since running our own node requires an upwards of around 400GB of storage, we will use a third-party plugin.

There are various plugins; for this tutorial, we will go with sauron. You can check the rest of the plugins here.

Setting Up Sauron

Core Lightning supports a plugin manager called reckless, which simplifies the installation and uninstallation of plugins with a single command. To install Sauron, execute the following command:

reckless install sauron

After running the above command, the sauron plugin will be downloaded in the ~/.lightning/reckless/sauron directory.

Reckless currently supports python plugins only

Running Core Lightning Node

To run the node on the testnet, execute the following command:

lightningd --testnet --disable-plugin bcli --plugin --plugin ~/.lightning/reckless/sauron/sauron.py --sauron-api-endpoint https://blockstream.info/testnet/api/

Congratulations! You now have a successfully running Lightning node with a third-party plugin acting as a Bitcoin node. You can interact with your node using lightning-cli.

Open a new terminal, SSH into the VM again, and run the following command to confirm whether you are able to interact with your node or not:

lightning-cli --testnet getinfo

You can check the various commands provided by lightning-clihere.