Chalkydri is a blazingly fast vision system for FRC written in Rust.

In development

Chalkydri isn't ready to use quite yet.

New here?

Check out our Getting Started guide!

About

Chalkydri was created by and is managed by FRC Team 4533 (Phoenix).

We're trying to make vision less of a black box, so all FRC teams can use it, no matter their size. We also want it to be easier to use effectively with less hassle.

Chalkydri has been built from the ground up for performance on popular FRC vision hardware and uses less CPU and memory than existing solutions.

Credits

• PhotonVision

  • Main inspiration for the design of Chalkydri

• Lincoln (Student, 4533)

  • Phoenix vision code lead

This is a generic guide. You probably want a guide specific to whatever device you're using:

• (Recommended) Raspberry Pi 3/4/5

If your device isn't listed here, it may work, but there's no guarantee. You can see our instructions for building from source if we aren't providing binaries for your target architecture.

Before we start, click here to start downloading our latest release. It might take a little while on poor connections.

Flashing your microSD card

We recommend Etcher, since it's easy to use and cross-platform.

It will figure everything out for you and flash Chalkydri.

That's it! Now you can learn how to use Chalkydri.

Chalkydri system design

Chalkydri has a somewhat complicated design, as most vision things do, but I really hate poor documentation.

Once the robot is powered on, each Chalkydri device will:

• Boot up (almost certainly faster than everything else, because Alpine is awesome like that)
• Attempt to connect to the roboRIO's NetworkTables server
• Initialize camera(s)
• Initialize ML accelerator(s) if applicable
• Prepare backends

Chalkydri waits until it connects to the NetworkTables server successfully to actually start running. It will negotiate with the roboRIO and start processing frames.

NetworkTables API

Chalkydri/
    Robot/
        Position/
            X
            Y
        Rotation
    Devices/
        1/
        2/
        3/
        ...

Chalkydri/Robot/

Topics relevant to the robot as a whole

• Chalkydri/Robot/Position/X (64-bit float) The robot's current X coord on the field
• Chalkydri/Robot/Position/Y (64-bit float) The robot's current Y coord on the field
• Chalkydri/Robot/Rotation (64-bit float) The robot's current rotation

Chalkydri/Devices/

Each device's device-specific topics are grouped under Chalkydri/Devices/{device id}/

• Chalkydri/Devices/X/Version (string) The device's Chalkydri version

Maintenance guide

This is for people working on Chalkydri itself

If you are a Chalkydri user, this isn't what you want.

Being a fairly advanced project as FRC goes, it's harder to just pick up and work on.

This is the incomplete guide to maintaining it, how the internals work, and the best learning resources.

• Setting up your development environment
• API documentation

The Chalkydri project has several subprojects, mostly libraries we needed to build it.

MiniNT

Our NetworkTables client implementation

The existing Rust implementation was overly complex and pulled in too many extra dependencies. We tried a fork, but ended up just implementing NT4 from scratch.

TFLedge

Our bindings to TensorFlow Lite and libedgetpu

Here's all the awesome learning resources I've found so far:

Getting started

• (Linux) Command Line for Beginners
• (Linux) The Linux Command Handbook
• (Rust) The Book
• (Rust) Rust by Example (interactive, web)
• (Rust) Rustlings (interactive, native)

Getting better

• (Rust) The rustdoc Book
• (Rust) The Performance Book
• mdBook Docs

Getting deeper

• The Rustonomicon (unsafe Rust code)

Chalkydri was built on and for Linux, so using other operating systems may be a little more difficult.

Install the Rust toolchain

It's highly recommended you use rustup to install the toolchain.

Build requirements

• A reasonably recent and powerful processor
• A reasonable amount of RAM
• Docker (if cross-compiling)
• A complete build requires a considerable amount of disk space:
  • 2GB (rec. 3GB): Chalkydri itself
  • 4GB (rec. 6GB): Bazel / libedgetpu
  • 2GB (rec. 8GB): TFLite
  • That totals up to a recommended 17GB chunk of your drive (or a minimum 8GB). The hope is most of this can happen in CI, so it's not a massive pain for everybody trying to work on Chalkydri.

Raspberry Pi

The Pi 3, 4, and 5 are supported.

Pi cameras

libcamera is required to interact with v3.

Coral Edge TPU

We made our own bindings to TFLite and libedgetpu.

Releases should be done with CI.

To release a new version, simply create a release on Github. CI will test and build the main branch.

AprilTags

We're using our own custom AprilTag library, chalkydri-apriltags, built from the ground up. We'll refer to it as "CAT".

API documentation (rustdoc)

Why?

We have a few reasons for writing our own, rather than using what's already out there. The reference C library:

• is very resource intensive
• uses a lot of advanced math which isn't covered in high school classes, making it harder for students to understand how it works
• has memory leaks (needs a citation)

Design overview

CAT is based on existing algorithms, but with some tweaks specific to our use case.

1. Get a frame
2. Grayscale & threshold
3. Detect corners
4. Cluster & filter outliers (TODO)
5. Find convex hulls
6. Check convex hulls
7. Decode tags

Grayscale & threshold

Converting RGB to grayscale and thresholding are combined into one step for performance.

I can't find the original reference I used for grayscale values.

For adaptive thresholding, "Bradley's method" seems like the best option. It's described in Adaptive Thresholding Using the Integral Image.

Corner detection

Corner detection is done using the aptly named FAST algorithm. Another advantage: it's very simple and easy to understand.

Cluster & filter outliers

OPTICS¹ is an algorithm for clustering and outlier detection.

It's similar to DBSCAN, which is described pretty well by Wikipedia:

given a set of points in some space, it groups together points that are closely packed (points with many nearby neighbors), and marks as outliers points that lie alone in low-density regions (those whose nearest neighbors are too far away).

You can read more about it in the Wikipedia article.

Find convex hulls

Gift wrapping algorithm

Check convex hulls

TODO

Decode tags

Decoding tags is done pretty much the same way the C library does it.

Important references

• Real-time Quadrilateral Object Corner Detection Algorithm Based on Deep Learning