Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Introduction

Turnkey is a toolchain management framework for Nix flakes that simplifies declaring and managing build tools in development environments.

What is Turnkey?

Turnkey bridges declarative TOML configuration with Nix package resolution, providing:

  • Simple Configuration: Declare toolchains in toolchain.toml
  • Reproducible Environments: Nix ensures consistent tool versions across machines
  • Incremental Builds: Fast, cached builds that only rebuild what changed
  • Language Support: Go, Rust, Python, TypeScript, Solidity, Jsonnet, and more

Key Features

  • Declarative toolchain management via TOML
  • Automatic dependency cell generation for the build system
  • Native tool wrappers with auto-sync (go, cargo, uv)
  • Modular Nix flake integration

Who Should Use This?

Turnkey is designed for teams who:

  • Want reproducible development environments
  • Need fast, incremental builds across multiple languages
  • Need to manage multiple language toolchains
  • Value declarative, version-controlled configuration

Next Steps

Why Turnkey

Modern software development faces a fundamental tension: we want the simplicity of working with familiar tools while also needing the reproducibility and scalability of sophisticated build systems.

Turnkey bridges this gap.

The Problem

Consider a typical development scenario. You have a project that uses Go, some Rust libraries, a Python testing framework, and TypeScript for the frontend. Each language has its own:

  • Package manager (go mod, cargo, pip/uv, npm/pnpm)
  • Build conventions
  • Test runners
  • IDE integrations

This works fine for small projects. But as projects grow, you encounter challenges:

  1. "Works on my machine" - Different developers have different tool versions
  2. Slow CI/CD - Every change rebuilds everything, even unrelated code
  3. Dependency hell - Conflicting versions across languages and packages
  4. AI agent friction - Automated tools struggle with slow, non-incremental builds

The enterprise answer to these problems is typically a monorepo with a sophisticated build system like Bazel or Buck2. But adopting a monorepo means:

  • Rewriting all your build logic
  • Learning new command-line tools
  • Breaking IDE integrations
  • Significant upfront investment

The Turnkey Solution

Turnkey takes a different approach: keep your familiar tools working normally while adding build system benefits invisibly.

# These still work exactly as expected
go build ./...
cargo test
pytest
npm run build

# But now you also have Buck2's power when you need it
buck2 build //...
buck2 test //...

The key insight is that most developers don't need to think about the build system most of the time. They want to:

  • Write code
  • Run tests
  • Get fast feedback

Turnkey provides this while maintaining a single source of truth for dependencies and builds that enables advanced features like:

  • Hermetic, reproducible builds
  • Incremental compilation across languages
  • Remote caching and execution
  • Atomic changes across the entire codebase

Who Is Turnkey For?

Turnkey is designed for teams that want:

Enterprise-grade infrastructure without abandoning their existing workflows. Your go build still works. Your IDE still works. Your junior developers don't need to learn build system internals to be productive.

A growth path from prototype to production. Start with normal language tooling. Adopt incremental build features as your needs grow. No big-bang rewrites.

AI-friendly development with fast feedback loops. AI coding assistants work better when builds are fast and incremental. Turnkey's caching means AI agents can iterate quickly.

Reproducibility without ceremony. Nix handles tool versioning. The build system handles caching. You focus on writing code.

The Turnkey Philosophy

  1. Tools should enhance, not replace - Native commands work normally
  2. Complexity should be opt-in - Start simple, add sophistication as needed
  3. Reproducibility is non-negotiable - Same inputs always produce same outputs
  4. Fast feedback enables better code - Incremental builds by default

In the following chapters, we'll explore the core principles that make this possible and how the architecture enables a seamless developer experience.

Core Principles

Turnkey is built on four core principles that guide every design decision. These principles often exist in tension with each other, and Turnkey's value lies in finding the right balance.

1. Native Tool Compatibility

Your existing commands should just work.

When you run go build, it should build your Go code. When you run cargo test, it should test your Rust code. LSP servers should provide autocomplete. IDEs should find definitions. This isn't a compromise - it's a requirement.

How It Works

Turnkey provides transparent wrappers (tw) around native tools that:

  • Pass through all commands unchanged by default
  • Watch for dependency file changes (go.mod, Cargo.lock, etc.)
  • Automatically regenerate build system dependency cells when needed
  • Never block or modify the developer's primary workflow
# The 'tw' wrapper is transparent
tw go get github.com/foo/bar    # Works exactly like 'go get'
                                 # But also updates build system deps if go.mod changed

# Or use 'go' directly - it still works
go build ./...                   # Normal Go build, no Buck2 involved

Why This Matters

  • Zero learning curve for basic workflows
  • IDE integrations continue working - gopls, rust-analyzer, pyright all function normally
  • Existing scripts and CI remain valid - no migration required
  • Developers stay in their comfort zone while infrastructure improves beneath them

2. Monorepo Benefits Without Monorepo Storage

Get unified versioning without storing the world in your repository.

Traditional monorepos store all code in one repository, enabling atomic changes and unified versioning. But this comes with costs:

  • Massive repository size
  • Complex code ownership
  • Slow git operations
  • Storage of third-party code

Turnkey provides the benefits of a monorepo without these costs through virtual cells.

How It Works

your-repo/
├── src/                    # Your source code
├── go.mod                  # Normal Go module
├── Cargo.toml              # Normal Cargo workspace
└── .turnkey/
    ├── godeps/            # Virtual cell: Go dependencies
    ├── rustdeps/          # Virtual cell: Rust dependencies
    └── prelude/           # Virtual cell: Build system prelude

The .turnkey/ directory contains cells - the build system's unit of code organization. These cells are:

  • Generated from your lock files (go.sum, Cargo.lock, etc.)
  • Deterministically reproducible via Nix
  • Treated as source code by the build system (enabling caching and incrementality)
  • Never committed to git (they're derived data)

The Result

  • Atomic changes across your code and its dependencies
  • Unified versioning - one lock file controls one version
  • Hermetic builds - Nix ensures reproducibility
  • Fast git operations - repository stays small

3. Incremental Build and Test

Only rebuild and retest what actually changed.

Modern CI/CD often wastes enormous resources rebuilding unchanged code. A small typo fix shouldn't trigger a full rebuild of the entire project.

How It Works

The incremental build system tracks fine-grained dependencies between:

  • Source files
  • Build rules
  • Test targets
  • Generated artifacts

When a file changes, the build system determines the minimal set of actions needed:

# Edit a single Go file
vim pkg/utils/helper.go

# The build system only rebuilds affected targets
tk build //...    # Rebuilds only what depends on helper.go
tk test //...     # Runs only tests that might be affected

Combined with remote caching, this means:

  • CI builds are fast because most artifacts are cached
  • Local builds benefit from CI's cached artifacts
  • AI agents can iterate quickly with sub-second feedback

Why This Matters for AI

AI coding assistants (like Claude Code) benefit enormously from fast builds:

  • Quick iterations mean more experiments per session
  • Fast test feedback enables test-driven development
  • Immediate error messages allow rapid course correction

Turnkey's incremental builds make AI-assisted development practical at scale.

4. Continuum of Experience

Scale from prototype to enterprise without rewrites.

Software projects exist on a spectrum:

StageNeeds
PrototypeQuick iteration, minimal ceremony
StartupFast CI, some reproducibility
GrowthCaching, parallelism, reliability
EnterpriseCompliance, governance, audit trails

Traditional build systems force you to choose: simple but limited, or powerful but complex. Turnkey provides a continuum:

Level 1: Just Use Native Tools

go build ./...
cargo test
pytest

No turnkey involvement. Everything works normally.

Level 2: Add Hermetic Tooling

# toolchain.toml
[toolchains]
go = {}
rust = {}
python = {}

Now your tools are versioned by Nix. "Works on my machine" disappears.

Level 3: Enable Incremental Builds

tk build //...
tk test //...

Get incremental builds and caching. CI becomes faster.

Level 4: Add Remote Caching

Share build artifacts across developers and CI. Builds that took 10 minutes now take 30 seconds.

Level 5: Remote Execution

Distribute builds across a cluster. Massive parallelism. Enterprise scale.

Why This Matters

You don't have to adopt everything at once. Start with Level 1 or 2. Move to higher levels as your needs grow. The underlying infrastructure supports your growth without requiring rewrites.


These four principles - native compatibility, virtual monorepo, incremental builds, and progressive adoption - form the foundation of Turnkey's design. In the next chapter, we'll see how the architecture implements these principles in practice.

Note: Turnkey currently uses Buck2 as its incremental build system. The architecture is designed to potentially support other build systems like Bazel in the future.

Architecture Overview

Turnkey combines three powerful technologies - Nix, an incremental build system, and devenv - into a cohesive developer experience. This chapter explains how these pieces fit together.

Current Implementation: Turnkey uses Buck2 as its incremental build system. The architecture is designed to potentially support other systems like Bazel in the future.

The Three Pillars

┌─────────────────────────────────────────────────────────────┐
│                     Developer Experience                    │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │ go build    │  │ tk build    │  │ IDE / LSP           │  │
│  │ cargo test  │  │ tk test     │  │ Autocomplete        │  │
│  │ pytest      │  │ tk run      │  │ Go to definition    │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                         Turnkey                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │ tw wrappers │  │ tk CLI      │  │ Dep generators      │  │
│  │ Auto-sync   │  │ Build wrap  │  │ godeps-gen, etc.    │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Core Technologies                        │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │    Nix      │  │Build System │  │      devenv         │  │
│  │ Hermetic    │  │ Incremental │  │ Shell environment   │  │
│  │ packages    │  │ builds      │  │ configuration       │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Nix: Hermetic Package Management

Nix provides reproducible package management. Every tool, compiler, and library has a precise version controlled by the flake.nix and flake.lock files.

What Nix provides:

  • Exact versions of go, cargo, python, node, etc.
  • System libraries and compilers
  • Build tools (buck2 itself)
  • Dependency fetching with verified hashes

Key benefit: When you enter the development shell, you have the exact same tools as every other developer and CI system.

Incremental Build System (Buck2)

The build system provides fast, incremental, and correct builds. It tracks dependencies at a fine-grained level and only rebuilds what's necessary.

What the build system provides:

  • Dependency tracking between files and targets
  • Parallel execution of independent tasks
  • Remote caching (share builds across machines)
  • Remote execution (distribute builds to a cluster)

Key benefit: After initial setup, builds are dramatically faster because unchanged code isn't rebuilt.

devenv: Developer Shell Configuration

devenv provides a declarative shell environment configured through Nix. It handles:

  • Environment variable setup
  • Shell hooks and initialization
  • Service management (databases, etc.)
  • Integration with direnv for automatic activation

Key benefit: Entering a project directory automatically sets up the complete development environment.

The Flow of Data

From Lock Files to Build System Cells

┌──────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│  go.mod/go.sum   │────▶│   godeps-gen     │────▶│  go-deps.toml    │
│  (native lock)   │     │  (generator)     │     │  (intermediate)  │
└──────────────────┘     └──────────────────┘     └──────────────────┘
                                                           │
                                                           ▼
┌──────────────────┐     ┌──────────────────┐     ┌──────────────────┐
│  .turnkey/godeps │◀────│      Nix         │◀────│  go-deps.toml    │
│  (Buck2 cell)    │     │  (fetcher)       │     │  (with hashes)   │
└──────────────────┘     └──────────────────┘     └──────────────────┘
  1. Native lock files (go.sum, Cargo.lock, pnpm-lock.yaml) define exact dependency versions
  2. Dependency generators (godeps-gen, rustdeps-gen, etc.) parse lock files and output intermediate TOML
  3. Nix fetches dependencies with verified hashes and creates build-system-compatible cells
  4. The build system treats these cells as source code, enabling full incrementality

The tw Wrapper Flow

Developer runs: tw go get github.com/foo/bar
                        │
                        ▼
              ┌─────────────────┐
              │  Snapshot state │  (hash go.mod, go.sum)
              └─────────────────┘
                        │
                        ▼
              ┌─────────────────┐
              │  Run go get     │  (native command)
              └─────────────────┘
                        │
                        ▼
              ┌─────────────────┐
              │ Check for diff  │  (did lock files change?)
              └─────────────────┘
                        │
              ┌─────────┴─────────┐
              ▼                   ▼
        [No change]         [Files changed]
              │                   │
              │                   ▼
              │         ┌─────────────────┐
              │         │ Run godeps-gen  │
              │         └─────────────────┘
              │                   │
              └───────────────────┘
                        │
                        ▼
                    [Done]

The tw wrapper ensures the build system's view of dependencies stays synchronized with native tools, without requiring developer intervention.

Directory Structure

A typical Turnkey-enabled project looks like:

project/
├── .buckconfig              → Symlink to generated config (Buck2)
├── .buckroot                → Marks project root for build system
├── .envrc                   → Activates devenv via direnv
├── flake.nix                → Nix flake configuration
├── flake.lock               → Locked Nix dependencies
├── toolchain.toml           → Turnkey toolchain declaration
│
├── src/                     → Your source code
│   ├── cmd/
│   ├── pkg/
│   └── rules.star           → Build rules
│
├── go.mod                   → Go module definition
├── go.sum                   → Go dependency lock
├── go-deps.toml            → Generated dependency manifest
│
├── Cargo.toml              → Rust workspace definition
├── Cargo.lock              → Rust dependency lock
├── rust-deps.toml          → Generated dependency manifest
│
└── .turnkey/               → Generated artifacts (gitignored)
    ├── prelude/            → Build system prelude cell
    ├── toolchains/         → Toolchain definitions
    ├── godeps/             → Go dependency cell
    ├── rustdeps/           → Rust dependency cell
    └── sync.toml           → Sync configuration

What's Committed to Git

  • Source code (src/)
  • Native project files (go.mod, Cargo.toml, etc.)
  • Lock files (go.sum, Cargo.lock, etc.)
  • Turnkey configuration (toolchain.toml)
  • Nix configuration (flake.nix, flake.lock)
  • Generated dependency manifests (go-deps.toml, etc.)

What's Generated (Not Committed)

  • .turnkey/ directory (regenerated from lock files)
  • .buckconfig (symlinked to Nix store, Buck2-specific)
  • Build outputs (buck-out/)

The Toolchain Flow

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│ toolchain.toml  │────▶│    Registry     │────▶│  Nix packages   │
│                 │     │   (mapping)     │     │                 │
│ [toolchains]    │     │ go = pkgs.go    │     │ /nix/store/...  │
│ go = {}         │     │ rust = pkgs...  │     │                 │
│ rust = {}       │     │                 │     │                 │
└─────────────────┘     └─────────────────┘     └─────────────────┘
                                                         │
                                                         ▼
                        ┌─────────────────┐     ┌─────────────────┐
                        │  Buck2 targets  │◀────│    mappings     │
                        │                 │     │                 │
                        │ toolchains//:go │     │ Generate rules  │
                        │ toolchains//... │     │ from registry   │
                        └─────────────────┘     └─────────────────┘
  1. toolchain.toml declares what toolchains you need
  2. The registry maps toolchain names to Nix packages
  3. mappings translate these into build system toolchain targets
  4. The build system uses the toolchain targets for builds

Summary

Turnkey's architecture achieves its goals through careful layering:

LayerResponsibilityTechnology
TopDeveloper UXNative tools, tw/tk wrappers
MiddleOrchestrationTurnkey, dependency generators
BottomExecutionNix (packages), build system (builds), devenv (shell)

Each layer can be understood independently, and the boundaries are clean enough that you can use partial features without understanding the whole system.

For detailed information about specific components, see the reference documentation.

Installation

Prerequisites

Before installing Turnkey, ensure you have:

  • Nix with flakes enabled
  • direnv (recommended) for automatic environment activation

Enabling Nix Flakes

If you haven't enabled flakes, add to ~/.config/nix/nix.conf:

experimental-features = nix-command flakes

Adding Turnkey to Your Project

New Project

Use the Turnkey template to create a new project:

nix flake init -t github:firefly-engineering/turnkey

Existing Project

Add Turnkey to your flake.nix inputs:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    turnkey.url = "github:firefly-engineering/turnkey";
  };

  outputs = { self, nixpkgs, turnkey, ... }: {
    # Your flake configuration
  };
}

Verifying Installation

After setup, enter the development shell:

nix develop

You should see the welcome message and have access to your declared toolchains.

Quick Start

This guide walks you through building your first project with Turnkey.

Create a toolchain.toml

Create a toolchain.toml file in your project root:

[toolchains]
go = {}

This declares that your project needs Go. Buck2 isn't declared: turnkey ships its own pinned buck2 release, which buck2.enable below adds to the shell.

Configure Your Flake

Update your flake.nix to use Turnkey:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
    turnkey.url = "github:firefly-engineering/turnkey";
    devenv.url = "github:cachix/devenv";
  };

  outputs = inputs@{ turnkey, devenv, ... }:
    turnkey.lib.mkFlake { inherit inputs; } {
      imports = [
        devenv.flakeModule
        turnkey.flakeModules.turnkey
      ];

      perSystem = { ... }: {
        turnkey.toolchains = {
          enable = true;
          declarationFiles.default = ./toolchain.toml;
          buck2.enable = true;
        };
      };
    };
}

Enter the Shell

nix develop

Build Something

Create a simple Go program and build it with Buck2:

tk build //path/to:target

Next Steps

Project Setup

This guide covers how to create a new Turnkey project or add Turnkey to an existing project.

New Project

Create a new Buck2 project using the Turnkey flake template:

mkdir my-project && cd my-project
nix flake init -t github:firefly-engineering/turnkey
direnv allow  # If using direnv

This creates:

  • flake.nix - Nix flake configuration with Turnkey enabled
  • toolchain.toml - Toolchain declaration (Go enabled by default)
  • .envrc - direnv configuration with symlink sync
  • .gitignore - Ignores Turnkey-managed files
  • rules.star - Root build file (template)

Existing Project

Add Turnkey to an existing Nix flake project:

1. Add Turnkey Input to flake.nix

{
  inputs = {
    # ... existing inputs ...
    turnkey.url = "github:firefly-engineering/turnkey";
  };

  outputs = inputs@{ flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      imports = [
        inputs.devenv.flakeModule
        inputs.turnkey.flakeModules.turnkey
      ];

      # ... rest of config ...

      perSystem = { pkgs, ... }: {
        turnkey = {
          enable = true;
          declarationFile = ./toolchain.toml;
        };

        devenv.shells.default = {
          turnkey.buck2.enable = true;
        };
      };
    };
}

2. Create toolchain.toml

[toolchains]
go = {}
# Add more as needed: rust, python, cxx

3. Update .gitignore

# Turnkey managed files
.buckconfig
.buckroot
.turnkey/
buck-out/

4. Create .envrc (if using direnv)

Turnkey provides a direnv library that handles all symlink management automatically:

use flake . --no-pure-eval

# Source the turnkey library and activate
source "$TURNKEY_DIRENV_LIB"
use_turnkey

The use_turnkey function handles:

  • Buck2 symlink management (.buckconfig, the prelude and toolchains cells) and the deps cells' directories
  • Re-evaluating the flake when a deps file changed since its cells were built
  • watch_file declarations for automatic reloads
  • Optional dependency file regeneration

Then allow it:

direnv allow

Directory Structure

A typical Turnkey project has this structure:

my-project/
├── .buckconfig           # Buck2 configuration (generated symlink)
├── .buckroot             # Empty file marking project boundary
├── .envrc                # direnv configuration
├── .turnkey/             # Generated cells (gitignored)
│   ├── prelude/          # Buck2 prelude
│   ├── toolchains/       # Language toolchains
│   ├── godeps/           # Go dependency cell (if configured)
│   └── rustdeps/         # Rust dependency cell (if configured)
├── flake.nix             # Nix flake configuration
├── flake.lock            # Locked dependencies
├── toolchain.toml        # Toolchain declarations
├── go-deps.toml          # Go dependencies (if using Go)
├── rust-deps.toml        # Rust dependencies (if using Rust)
└── rules.star            # Root build file

Generated Files

When you enter the devenv shell, Turnkey generates:

FileDescription
.buckconfigSymlink to Nix-managed Buck2 configuration
.buckrootEmpty file marking project boundary
.turnkey/toolchainsSymlink to generated toolchains cell
.turnkey/godepsGo dependencies cell, a directory tk materialize maintains (if configured)
.turnkey/preludeSymlink to the prelude

The Prelude

The prelude is always turnkey's: the prelude built with turnkey's pinned buck2 release, with turnkey's patches and extensions applied. It isn't configurable, for the same reason buck2 itself isn't: they are one release (ADR 0002).

If you really need a prelude of your own, prelude.path takes a derivation or a path. That is off the supported path, and it turns test result caching off, since turnkey can't know which of that prelude's test rules are cache-safe:

turnkey.toolchains.buck2.prelude.path = ./my-prelude;

direnv Integration

For automatic environment activation with full Turnkey support, create .envrc:

use flake . --no-pure-eval

source "$TURNKEY_DIRENV_LIB"
use_turnkey

Then allow it:

direnv allow

use_turnkey runs tk sync to regenerate stale dependency files (the same rules as tk sync everywhere else, from .turnkey/sync.toml), keeps the cells current and watches the rules' files so that a changed deps file reloads the shell. When a deps file's content differs from the one the shell's cells were built from (regenerated by that tk sync, or earlier by tw or a tk sync of your own), it evaluates the flake again and loads the .envrc once more, so the cells are rebuilt in the same load. It takes options:

  • use_turnkey --skip-regen - Skip dependency file regeneration
  • use_turnkey --skip-sync - Skip symlink synchronization
  • use_turnkey --only-<rule> / --skip-<rule> - Sync only, or all but, the named deps rules (go, rust, pylock, python, javascript, solidity)
  • Environment variables like TURNKEY_SKIP_ALL=1 (with TURNKEY_ENABLE_<RULE>=1 to add rules back) or TURNKEY_SKIP_<RULE>=1

Buck2 Configuration

The .buckconfig is generated automatically. For project-specific settings, create .buckconfig.local:

[build]
# Custom build settings

[project]
# Project-specific settings

Verifying Setup

After entering the shell, verify Buck2 is configured:

# Check toolchains
buck2 targets toolchains//...

# Run Go via toolchain
buck2 run toolchains//:go[go] -- version

# Build a target
buck2 build //...

Common Issues

If .turnkey/ symlinks aren't created:

  1. Check that you're using direnv or manually sourcing the environment
  2. Verify environment variables are set:
    echo $TURNKEY_BUCK2_CONFIG
    echo $TURNKEY_BUCK2_TOOLCHAINS_CELL
    
  3. Re-allow direnv:
    direnv allow
    

Buck2 Can't Find Cells

If Buck2 reports missing cells:

  1. Check .buckconfig is a valid symlink:
    ls -la .buckconfig
    
  2. Verify cell paths in .buckconfig exist:
    cat .buckconfig
    
  3. Ensure you've entered the Nix shell:
    nix develop
    

toolchain.toml

The toolchain.toml file declares which toolchains your project needs.

Basic Structure

[toolchains]
go = {}
rust = {}
python = {}

Each key under [toolchains] is a toolchain name that will be resolved from the registry.

buck2 is not declared here. Turnkey ships its own pinned buck2 release to the shells that have the Buck2 integration, and declaring buck2 or buck2-toolchain is an error. See The buck2 version.

Version Pinning

You can pin specific versions when the registry provides multiple versions:

[toolchains]
go = { version = "1.22" }      # Pin to Go 1.22
python = { version = "3.11" }  # Pin to Python 3.11
rust = {}                       # Use registry default

If no version is specified, the registry's default version is used.

Available Toolchains

Languages

  • go - Go compiler and tools
  • rust - Rust compiler (rustc)
  • cargo - Cargo package manager
  • clippy - Rust linter
  • rustfmt - Rust formatter
  • rust-analyzer - Rust LSP server
  • python - Python interpreter
  • uv - Python package manager
  • ruff - Python linter and formatter
  • nodejs - Node.js runtime
  • typescript - TypeScript compiler
  • biome - Fast linter/formatter for JS/TS/JSON

Solidity

  • solc - Solidity compiler
  • foundry - Ethereum dev toolkit (forge, cast, anvil)

Other Tools

  • nix - Nix package manager
  • reindeer - Rust Buck2 target generator
  • jsonnet - Jsonnet to JSON compiler
  • mdbook - Documentation tool
  • tk - Turnkey CLI wrapper for buck2

Internal Tools

Dependency generators (godeps-gen, rustdeps-gen, pydeps-gen, jsdeps-gen, soldeps-gen) are automatically included when their corresponding language is enabled. You don't need to list them in toolchain.toml.

For example, if you have go = {} in your toolchain.toml and buck2.go.enable = true in your flake.nix, godeps-gen will automatically be available in your shell.

Example Configurations

Minimal Go Project

[toolchains]
go = {}

Full-Stack Project

[toolchains]
# Backend
go = {}
python = {}

# Frontend
nodejs = {}
typescript = {}
biome = {}

# Development
nix = {}

Pinned Versions

[toolchains]
go = { version = "1.22" }
python = { version = "3.11" }
nodejs = { version = "20" }
rust = { version = "1.75" }

Custom Registries

The registry mapping toolchain names to packages can be customized in your flake.nix. See Registry Pattern for details on:

  • Adding custom toolchains via registryExtensions
  • Creating reusable registry overlays
  • Multi-version toolchain support

Buck2 Integration

Turnkey provides first-class Buck2 integration with automatic toolchain and dependency cell generation.

Enabling Buck2

In your flake.nix:

turnkey.toolchains = {
  enable = true;
  declarationFiles.default = ./toolchain.toml;
  buck2.enable = true;
};

By default only the default shell gets the Buck2 integration: the pinned buck2 on its PATH, the prelude, and the generated cells. List other shells in buck2.shells to give them the integration too:

turnkey.toolchains = {
  declarationFiles = {
    default = ./toolchain.toml;
    ci = ./toolchain.ci.toml;
    docs = ./docs/toolchain.toml;   # no Buck2 here
  };
  buck2 = {
    enable = true;
    shells = [ "default" "ci" ];
  };
};

Naming a shell that declarationFiles doesn't define is an error.

The buck2 version

Each turnkey revision ships exactly one buck2 release, together with the prelude built with it. You don't choose buck2 in toolchain.toml: you get a newer buck2 by updating your turnkey input (nix flake update turnkey). Turnkey's test runner speaks buck2's internal test protocol and its prelude is patched for one prelude revision, so each only works against the release it was built for (ADR 0002).

To see which release you have, read turnkey.toolchains.buck2.version, or look at the shell's welcome message, which ends with (buck2 <version>) when a welcomeMessage is set.

Migrating from a declared buck2

Earlier turnkey revisions took buck2 from toolchain.toml. Declaring it now fails evaluation:

error: turnkey: /nix/store/…-source/toolchain.toml declares buck2-toolchain, but turnkey now ships buck2 itself.

To migrate:

  1. Remove the buck2 or buck2-toolchain entry from every toolchain.toml.
  2. Keep buck2.enable = true; in flake.nix. If a shell other than default used to declare buck2, add its name to buck2.shells. Before, declaring buck2 is what gave a shell the Buck2 integration.
  3. buck2-toolchain also carried reindeer. If you use it, declare reindeer = {} in toolchain.toml.

To run a different buck2 anyway, override turnkey's toolbox input with follows. This is unsupported: test result caching or the prelude may break against another release.

Generated Cells

When Buck2 integration is enabled, Turnkey generates:

Toolchains Cell

Located at .turnkey/toolchains/, contains toolchain rules for each declared language:

  • toolchains//:go - Go toolchain
  • toolchains//:rust - Rust toolchain
  • toolchains//:python - Python toolchain
  • etc.

It also holds toolchains//conditions:<os>-<cpu>, one config_setting per platform in buck2.platforms (the flake's systems by default), combining its OS and CPU constraints. Rules sync keys a select() on one when deps differ by CPU within one OS (Platform-Conditional Deps). With buck2.go.allowedBuildTags, which also sets .buckconfig's go.allowed_build_tags, it holds the settings combining the platform's OS and CPU with each allowed tag, set (linux-x86_64-integration) or unset (linux-no_integration), for a Go library whose imports depend on a tag.

Prelude Cell

The Buck2 prelude is provided via Nix at .turnkey/prelude/: the prelude built with turnkey's pinned buck2 release, with turnkey's patches and extensions applied.

The prelude and toolchains cells are symlinks into the Nix store. After either changes, a plain buck2 call needs a tk call or a buck2 kill first: see Symlinked Cells and Plain buck2.

A Prelude of Your Own

prelude.path replaces turnkey's prelude with a derivation or a path. It is off the supported path, and it turns test result caching off:

turnkey.toolchains.buck2.prelude.path = ./my-prelude;

Directories Buck2 Doesn't See

The generated .buckconfig's project.ignore lists directories, relative to the project root, that buck2 skips: //... doesn't load their rules.star files, and buck2's file watcher drops their events, so they never show up as File changed: lines. turnkey always ignores two kinds of directory that no build reads but something writes to all the time:

  • the VCS's metadata: .git, .jj, .hg, .sl;
  • the devenv and direnv state: .devenv, .direnv.

ignore adds to them. Use it for trees that are projects of their own, such as test fixtures whose targets only build in their own checkout:

turnkey.toolchains.buck2.ignore = [ "e2e/fixtures" ];

Buck2 reads project.ignore when its daemon starts, so a change takes effect once the daemon restarts. .buckconfig is a store symlink, so tk restarts it on its next command; plain buck2 needs a buck2 kill first (Symlinked Cells and Plain buck2).

Dependency Cells

Language-specific dependency cells are generated when configured:

  • godeps// - Go dependencies from go-deps.toml
  • rustdeps// - Rust dependencies from rust-deps.toml
  • pydeps// - Python dependencies from python-deps.toml
  • jsdeps// - JavaScript dependencies from js-deps.toml
  • soldeps// - Solidity dependencies from solidity-deps.toml

They are write-once directories, which plain buck2 reads without a daemon restart.

See Managing Dependencies for configuration details.

The .turnkey Directory

Turnkey uses a .turnkey directory in your project root to store build artifacts, caches, and generated cells. This convention provides automatic isolation from language toolchains.

Why .turnkey?

The .turnkey directory serves as the isolation directory for Buck2 builds. By using a dot-prefixed name, we get automatic exclusion from most language toolchains:

ToolBehaviorConfiguration Needed
GoIgnores directories starting with . or _None (built-in)
CargoDoesn't auto-discover crates in dot directoriesNone (built-in)
pytestAutomatically ignores dot directoriesNone (built-in)
JestRequires explicit configurationYes
VitestRequires explicit configurationYes

This means Go won't try to compile generated Buck2 cells, Cargo won't discover them as workspace members, and pytest won't scan them for tests.

Directory Structure

.turnkey/
├── books/           # mdbook serve output (gitignored)
├── prelude/         # Symlink to Buck2 prelude derivation
├── toolchains/      # Symlink to generated toolchains cell
├── godeps/          # Real directory: the write-once Go cell (tk materialize)
│   ├── .buckconfig
│   ├── .deps-file-sha256   # the go-deps.toml it was built from
│   ├── _store/<store path name>  # one symlink per module, never retargeted
│   └── vendor/<import path>/rules.star  # an alias package per Go package
├── godeps.lock      # held while tk materialize runs
├── jsdeps/          # Real directory: the write-once JavaScript cell (tk materialize)
│   ├── .buckconfig
│   ├── .deps-file-sha256   # the js-deps.toml it was built from
│   ├── rules.star          # an instance per pnpm snapshot, an alias per direct dependency
│   ├── _store/<store path name>  # one symlink per package, never retargeted
│   └── vendor/<name>@<version>/rules.star  # an alias package per package
├── jsdeps.lock      # held while tk materialize runs
├── pydeps/          # Real directory: the write-once Python cell (tk materialize)
│   ├── .buckconfig
│   ├── .deps-file-sha256   # the python-deps.toml it was built from
│   ├── _store/<store path name>  # one symlink per distribution, never retargeted
│   └── vendor/<name>/rules.star  # an alias package per distribution
├── pydeps.lock      # held while tk materialize runs
├── rustdeps/        # Real directory: the write-once Rust cell (tk materialize)
│   ├── .buckconfig
│   ├── .deps-file-sha256   # the rust-deps.toml it was built from
│   ├── _store/<store path name>  # one symlink per crate, never retargeted
│   └── vendor/<crate>@<version>/rules.star, vendor/<crate>/rules.star  # aliases
├── rustdeps.lock    # held while tk materialize runs
├── soldeps/         # Real directory: the write-once Solidity cell (tk materialize)
│   ├── .buckconfig
│   ├── .deps-file-sha256   # the solidity-deps.toml it was built from
│   ├── rules.star          # bundle, and an alias per package
│   ├── _store/<store path name>  # one symlink per package, never retargeted
│   └── vendor/<name>/rules.star  # an alias package per package, beside
│                                 # links to its files for native forge
├── soldeps.lock     # held while tk materialize runs
├── gcroots/godeps, gcroots/jsdeps, gcroots/pydeps, gcroots/rustdeps, gcroots/soldeps # GC roots for the cells' current indexes
├── .cell-targets    # the store symlinks' targets, for tk's cell-freshness check
├── .cell-targets.<isolation dir>  # the same, for another isolation directory's daemon
├── edits/, patches/ # tk compose's edits and generated patches
└── sync.toml        # Symlink to the rules tk sync follows

The prelude and toolchains cells are symlinks to Nix store paths containing the generated Buck2 cells. The Go, Rust, Python, Solidity and JavaScript cells are real directories that tk materialize, run by the shell, keeps in line with the cell index Nix builds: a dependency change rewrites only the entries for the modules, crates, distributions or packages that changed (ADR 0004, ADR 0008, ADR 0010, ADR 0011, ADR 0012). Don't edit it; the shell rewrites it on every load.

Symlinked Cells and Plain buck2

Two cells are symlinks into the Nix store, repointed when the shell builds a new one:

  • .turnkey/prelude, the prelude;
  • .turnkey/toolchains, the toolchains cell.

.buckconfig and .turnkey/sync.toml are store symlinks too. They stay symlinks: they change only when the shell is rebuilt (a turnkey upgrade, a nix flake update, a toolchain.toml or flake.nix edit), which reloads it.

The deps cells, rustdeps, godeps, pydeps, jsdeps and soldeps, are the write-once directories above, which moved off symlinks in #234: a dependency change never repoints a link, so any caller, plain buck2 included, reads the new version. A deps cell you set yourself with buck2.<language>.cell, a derivation without a cell index, is still a symlink.

A running daemon doesn't notice a repointed symlink. It keeps the build files and sources it already read through the old target, and reads the packages it hadn't loaded from the new one. A build can then mix the old and the new cell, with no error.

tk checks for it. Before each command it syncs for (every buck2 command but the pass-through ones), tk reads the targets of .buckconfig and of every entry of .turnkey/ that is a symlink into the store, and compares them with those it saved in .turnkey/.cell-targets. When one was added, removed or repointed, it prints tk: cell symlink changed, restarting buck2 daemon (unless --quiet), runs buck2 kill, and saves the new targets. The first run only saves them. The new daemon starts without the old one's state, so the next build re-runs its actions.

Each isolation directory has a daemon of its own, so with --isolation-dir, tk checks against that directory's state file instead, .turnkey/.cell-targets.<dir> (.cell-targets.turnkey-ci for tk --isolation-dir=ci), and restarts that directory's daemon: buck2 --isolation-dir .turnkey-ci kill. A change one daemon was restarted for still restarts each other one the next time tk runs against it.

The check doesn't cover:

  • plain buck2: a script, a tool that runs buck2 itself, or a shell with TURNKEY_NO_ALIAS=1. The shell's buck2 alias for tk only applies to the interactive shell;
  • tk --no-sync, which skips it with the sync;
  • an isolation directory set only through BUCK_ISOLATION_DIR: without --isolation-dir, tk restarts the daemon buck2 picks from the environment, but checks it against .turnkey/.cell-targets, the state of the shell's own .turnkey daemon. Pass --isolation-dir instead.

So after the shell is rebuilt, before a plain buck2 call, either:

  • run a tk command that syncs, such as tk build, which restarts the daemon if a symlink changed; or
  • run buck2 kill, with the same --isolation-dir as the call.

In turnkey's own repository, these run plain buck2:

  • the e2e tests (e2e/tests/), which call buck2 build, test and run in a fixture project's shell;
  • scripts/ci-smoke.sh, whose integration level builds and tests each language's example;
  • check-test-caching (src/cmd/check-test-caching/__main__.py), which runs buck2 bxl;
  • the starlark-lint git hook, buck2 --isolation-dir .turnkey-lint starlark lint. It has a daemon of its own, which tk never restarts: after a turnkey upgrade, run buck2 --isolation-dir .turnkey-lint kill.

Buck2 Configuration

The .buckconfig sets the isolation directory:

[buck2]
isolation_dir = .turnkey

This tells Buck2 to store all build outputs under .turnkey/buck-out/ instead of the default buck-out/.

The tk Command

The tk command wraps buck2 and automatically translates the --isolation-dir flag to use .turnkey-prefixed directories:

# These are equivalent:
tk --isolation-dir=foo build //...
buck2 --isolation-dir=.turnkey-foo build //...

This allows multiple isolated builds while maintaining the dot-prefix convention.

JavaScript/TypeScript Configuration

Unlike Go, Cargo, and pytest, JavaScript test runners need explicit configuration to ignore dot directories.

Jest

Add to your jest.config.js:

module.exports = {
  testPathIgnorePatterns: [
    '/node_modules/',
    '/buck-out/',
    '/\\.'  // Ignore all dot-prefixed directories
  ],
};

Or in package.json:

{
  "jest": {
    "testPathIgnorePatterns": [
      "/node_modules/",
      "/buck-out/",
      "/\\."
    ]
  }
}

Vitest

Add to your vitest.config.ts:

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    exclude: [
      '**/node_modules/**',
      '**/buck-out/**',
      '**/.*/**'  // Ignore all dot-prefixed directories
    ],
  },
});

Migration Notes

If you're migrating from a project that used buck-out/ directly:

  1. One-time cache invalidation: Buck2 caches are stored per isolation directory. Switching to .turnkey means a clean rebuild on first run.

  2. Update .gitignore: Ensure .turnkey/ is in your .gitignore:

    .turnkey/
    
  3. Update CI scripts: If CI scripts reference buck-out/, update them to .turnkey/buck-out/.

Multiple Isolation Directories

For advanced use cases (parallel builds, different configurations), you can use multiple isolation directories:

# Development build
tk build //...

# Release build with different isolation
tk --isolation-dir=release build //...
# Creates .turnkey-release/

# CI build
tk --isolation-dir=ci build //...
# Creates .turnkey-ci/

Each isolation directory maintains its own:

  • Buck2 daemon
  • Build cache
  • Output artifacts

This is useful for:

  • Running multiple Buck2 daemons simultaneously
  • Keeping CI caches separate from local development
  • Testing different build configurations

Shell Environment

Turnkey configures your development shell with all declared toolchains.

Environment Variables

When entering the shell, Turnkey sets:

  • PATH - Includes all toolchain binaries
  • TURNKEY_DIRENV_LIB - Path to direnv integration library

direnv Integration

For automatic shell activation, use direnv with .envrc:

use flake

Shell Entry Hooks

Turnkey performs these actions on shell entry:

  1. Symlinks .turnkey/prelude to the prelude cell
  2. Symlinks .turnkey/toolchains to the generated toolchains
  3. Brings the dependency cells' directories in line with their cell indexes (tk materialize)
  4. Displays welcome message (if configured)

Verbose Mode

For debugging, set TURNKEY_VERBOSE=1:

TURNKEY_VERBOSE=1 nix develop

Multiple Shells

You can define multiple shells with different toolchains:

turnkey.toolchains.declarationFiles = {
  default = ./toolchain.toml;
  ci = ./toolchain.ci.toml;
};

Access with:

nix develop .#ci

IDE Integration

This guide explains how to configure your IDE to work seamlessly with Turnkey's automatic dependency synchronization.

Overview

Turnkey can automatically update rules.star files when you modify source code imports. While this happens automatically when running tk build, you can also configure your IDE to trigger sync on file save for immediate feedback.

VS Code

Run on Save Extension

Install the Run on Save extension, then add to your workspace .vscode/settings.json:

{
  "emeraldwalk.runonsave": {
    "commands": [
      {
        "match": "\\.(go|rs|py|ts|tsx|sol)$",
        "cmd": "tk rules sync --quiet ${fileDirname}"
      }
    ]
  }
}

This runs tk rules sync on the directory containing the modified file whenever you save a source file.

Task-based Approach

Alternatively, create a VS Code task in .vscode/tasks.json:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Sync rules.star",
      "type": "shell",
      "command": "tk rules sync",
      "presentation": {
        "reveal": "silent",
        "panel": "shared"
      },
      "problemMatcher": []
    }
  ]
}

Then bind it to a keyboard shortcut in keybindings.json:

{
  "key": "ctrl+shift+s",
  "command": "workbench.action.tasks.runTask",
  "args": "Sync rules.star"
}

JetBrains IDEs (IntelliJ, GoLand, PyCharm, etc.)

File Watchers

  1. Go to Settings > Tools > File Watchers

  2. Click + to add a new watcher

  3. Configure:

    • Name: Turnkey Rules Sync
    • File type: Go files (or your language)
    • Scope: Project Files
    • Program: tk
    • Arguments: rules sync --quiet $FileDir$
    • Output paths to refresh: $FileDir$/rules.star
    • Working directory: $ProjectFileDir$
  4. Under Advanced Options:

    • Check: "Trigger the watcher on external changes"
    • Uncheck: "Auto-save edited files to trigger the watcher"

External Tools

Alternatively, set up an external tool:

  1. Go to Settings > Tools > External Tools

  2. Click + to add:

    • Name: Sync rules.star
    • Program: tk
    • Arguments: rules sync
    • Working directory: $ProjectFileDir$
  3. Assign a keyboard shortcut in Keymap settings

Neovim

Add to your Neovim configuration:

-- Auto-run tk rules sync on save for supported file types
vim.api.nvim_create_autocmd("BufWritePost", {
  pattern = { "*.go", "*.rs", "*.py", "*.ts", "*.tsx", "*.sol" },
  callback = function()
    local file_dir = vim.fn.expand("%:p:h")
    vim.fn.jobstart({ "tk", "rules", "sync", "--quiet", file_dir }, {
      on_exit = function(_, code)
        if code ~= 0 then
          vim.notify("tk rules sync failed", vim.log.levels.WARN)
        end
      end,
    })
  end,
})

Emacs

Add to your Emacs configuration:

(defun turnkey-sync-rules ()
  "Run tk rules sync on the current file's directory."
  (when (and buffer-file-name
             (string-match-p "\\.\\(go\\|rs\\|py\\|ts\\|tsx\\|sol\\)$" buffer-file-name))
    (let ((default-directory (file-name-directory buffer-file-name)))
      (start-process "tk-rules-sync" nil "tk" "rules" "sync" "--quiet" "."))))

(add-hook 'after-save-hook #'turnkey-sync-rules)

Configuration Options

Module Options

Rules sync is configured through turnkey's Buck2 options in your flake.nix, which generate the [rules] section of .turnkey/sync.toml (a generated file: don't edit it):

turnkey.toolchains.buck2.rules = {
  enabled = true;    # Enable rules.star sync (default: false)
  autoSync = true;   # Auto-sync before tk build (default: true)
  strict = false;    # Fail if rules would change - for CI (default: false)
};

Sync finds each language's internal targets from its own manifest (go.mod, Cargo.toml, the uv workspace) and uses turnkey's deps cells (godeps, rustdeps, pydeps, jsdeps, soldeps), through the deps file each is built from (the language's depsFile). Both reach sync through the [[languages]] of .turnkey/sync.toml. The platforms it resolves deps for come from buck2.platforms (see Platform-Conditional Deps).

Command Line Options

tk rules sync              # Sync only stale files (git-based detection)
tk rules sync --force      # Force sync all files
tk rules sync --verbose    # Show detailed output
tk rules sync --dry-run    # Show what would change without writing
tk rules check             # Check every rules.star file (exit 1 if any is stale)

Staleness Detection

tk rules sync and the sync tk runs before buck2 commands skip work that can't have changed:

  1. Git-based: only directories with uncommitted source file changes are considered
  2. Mtime-based: within those, a rules.star newer than every source file next to it is skipped

--force (or --all) turns both off. This means tk rules sync is nearly instant in most cases, making it suitable for on-save hooks.

tk rules check uses neither: it always checks every rules.star. Once a stale rules.star is committed, git reports no change for it and nothing makes its sources newer, so a filtered check would pass it forever.

Preservation Markers

If you have manual dependencies that shouldn't be auto-managed, use preservation markers:

go_binary(
    name = "my-app",
    srcs = ["main.go"],
    deps = [
        # turnkey:auto-start
        "godeps//vendor/github.com/google/uuid:uuid",
        # turnkey:auto-end
        # turnkey:preserve-start
        # Manual override for special case
        "//special:dep",
        # turnkey:preserve-end
    ],
)

Dependencies between preserve-start and preserve-end markers are never modified by sync.

Opting a Target Out

To keep sync away from one target entirely, for example to work around a problem, put a # turnkey:no-sync comment on its own line right before the rule:

# Links a hand-built native library sync knows nothing about
# turnkey:no-sync
rust_library(
    name = "my-lib-native",
    deps = _COMMON_DEPS + ["//third-party/native:lib"],
)

Sync never changes an opted-out target. tk rules sync -v and tk rules check -v list them as OPTED OUT:.

Platform-Conditional Deps

Some deps are only needed on some platforms. Sync resolves every target's deps on each platform the project builds for, so what it writes is the same whichever machine runs it. The platforms are buck2.platforms, the flake's systems by default:

turnkey.toolchains.buck2.platforms = [ "x86_64-linux" "aarch64-darwin" ];

They reach sync through the [conditions] section of .turnkey/sync.toml, in Buck2's names:

[conditions]
settings = "toolchains//conditions"

[[conditions.platforms]]
os = "linux"
cpu = "x86_64"

Deps every platform needs are written as a plain list. The others are written as a select() after it:

rust_library(
    name = "my-lib",
    deps = [
        # turnkey:auto-start
        "rustdeps//vendor/libc:libc",
        # turnkey:auto-end
    ] + select({
        "config//os:linux": ["rustdeps//vendor/fuser:fuser"],
        "config//os:macos": [],
    }),
)
  • A key is the smallest one that says exactly where the deps apply: config//os:<os> when they differ only by OS (or config//cpu:<cpu> by CPU alone), otherwise one of the toolchains cell's config_settings combining both, toolchains//conditions:<os>-<cpu>, one per platform. Go build tags are dimensions too (see Go below), combined the same way.
  • Every platform gets a branch, empty if it needs nothing more, and there is no DEFAULT: building for a platform that isn't listed fails instead of silently missing deps.
  • Sync reads this form back and owns all of it. The turnkey:auto and turnkey:preserve markers apply to the plain list only. A change to one branch rewrites only that branch.
  • A target whose deps don't depend on the platform keeps a plain list.

A select() sync can't read (its keys aren't config//os:*, config//cpu:*, toolchains//conditions:* or DEFAULT, or its values aren't lists of labels) makes the target unreadable, like any other expression.

Where Deps Come From

  • Rust: the crate's Cargo.toml, not its sources. [dependencies] go to library and binary targets; test targets get [dependencies] plus [dev-dependencies]. workspace = true entries are resolved against the root Cargo.toml, a workspace member maps to its own target, and any other crate maps to rustdeps//vendor/<package>:<package> (a renamed dependency maps to its package). An existing label that pins a version (rustdeps//vendor/tokio@1.50.0:tokio) satisfies the unversioned one. A target-specific table ([target.'cfg(...)'.dependencies], or a target triple) applies on the platforms its spec holds on, so its deps are platform-conditional: a dep every platform gets is a plain dep, one no platform gets is dropped. cfg() supports target_os, target_family (unix), target_arch, target_pointer_width, target_env, target_vendor, target_endian and all/any/not. Build dependencies are not synced: sync reports them and leaves any existing dep on them alone. See Rust Features for features, optional dependencies and dependencies on a member's variant.
  • Go: the imports go list reports, on every platform: it runs once per platform with GOOS, GOARCH and CGO_ENABLED=1 set, so a _linux.go file's imports become a config//os:linux branch whichever machine runs sync. A binary or a test is built with its build_tags, taken literally (none is a plain go build). A library gets its tags from the configuration, through the prelude's transition: each tag in buck2.go.allowedBuildTags that its build constraints use is an on/off dimension, and imports that depend on one become a select() on prelude//go/tags/constraints:<tag>[set]/[unset], or on a toolchains cell setting combining it with the platform (toolchains//conditions:linux-<tag>, ...-no_<tag>). Test targets get _test.go imports, external test packages' (XTestImports) included.
  • Python: the imports found in the sources. An import of a package that a uv workspace member provides (turnkey.cfg, from turnkey import cfg) maps to that member's target. The packages come from the members listed in the root pyproject.toml's [tool.uv.workspace] and their source layout, so a downstream namespace such as acme.* works the same way. Any other import maps to pydeps.
  • TypeScript: the npm packages imported by the sources, written to the target's npm_deps as the jsdeps cell's jsdeps//:<npm name> aliases (jsdeps//:@types/lodash), plus each one's @types/... package when it is a direct dependency too. Only the root package.json's dependencies (js-deps.toml's [direct]) map: an import of anything else is unmapped. The target's deps (other TypeScript targets) are not synced.
  • Other languages: the imports found in the sources, mapped to targets.

Sync never removes a dep it can't account for:

  • If a target has imports sync can't map to a target (reported as unmapped import), its deps are incomplete, so sync adds what it resolved but removes nothing, and prints a KEPT: line naming the deps it kept and why.
  • A target whose deps is an expression (_DEPS, _COMMON_DEPS + [...]) rather than a list of labels is not synced. Unless # turnkey:no-sync opts it out, tk rules sync and tk rules check report it as UNREADABLE:, and it makes tk rules check and strict mode fail: write its deps as a list of labels, or opt it out. (The sync before a build doesn't report it.)
  • Deps are only added or removed, never reordered.

Rust Features

A Rust target builds what Cargo would, and sync keeps it that way: it writes the target's features (the literal list the prelude passes to rustc, one --cfg feature="..." each) as well as its deps.

  • A primary target, one that sets neither of the attributes below, builds what cargo build -p <crate> builds: the crate's default features, expanded. An optional dependency is a dep only when an enabled feature activates it (dep:x, an implicit feature, x/feat).

  • A variant asks for features in Cargo's terms, on two attributes of turnkey's prelude Rust rules that rustc never sees:

    rust_library(
        name = "composition-full",
        crate = "composition",
        cargo_features = ["watcher"] + select({
            "config//os:linux": ["fuse"],
            "config//os:macos": ["fuse-t"],
        }),
        # default_features = False,  # as Cargo's default-features
    )
    

    Sync expands the request as Cargo would for a dependency asking for those features (dep:x, x/feat, weak x?/feat, feature-to-feature, and default unless default_features = False), and writes the features and deps it gives, per platform when the request is a select().

  • features is literal: default is written only when listed, and Buck2 adds nothing implicit.

  • A dependency on a workspace member that asks for features (its own features = [...] with the [workspace.dependencies] entry's, those its crate's features forward to it, and the member's defaults unless default-features = false) maps to the member's rust_library whose request enables exactly the same features, per platform. If none or several do, the dependency is reported as an unmapped import naming the features it needs, and nothing is removed.

A crate that doesn't build under Buck2 is a bug to fix in the rustdeps cell, not a reason to make a target a variant.

rust-analyzer sees a select()'d features through the host's branch.

Troubleshooting

Sync not running

  1. Ensure deps-extract is in your PATH (built with cargo install --path src/rust/deps-extract)
  2. Check that [rules] enabled = true in .turnkey/sync.toml
  3. Verify the file type is supported (Go, Rust, Python, TypeScript, Solidity)

Sync too slow

  1. Use the default staleness detection (don't use --force in on-save hooks)
  2. Target a specific directory: tk rules sync src/cmd/myapp

Wrong dependencies detected

  1. Check your *-deps.toml files are up to date (run tk sync)
  2. Verify internal prefix configuration in sync.toml
  3. Run tk rules sync --verbose to see what's being detected

FUSE Composition Layer

The FUSE composition layer provides a unified filesystem view of your repository and its dependencies at a fixed mount location. This enables:

  • Predictable paths for remote cache compatibility
  • Transparent editing of external dependencies
  • Automatic consistency management during updates

Quick Start

Manual (ad-hoc)

# Start the daemon for a single repo
turnkey-composed start --mount-point ~/firefly/turnkey --repo-root . --backend fuse

# Work from the mount point
cd ~/firefly/turnkey
buck2 build root//...

# Stop
turnkey-composed stop
# Install the service (runs on login)
turnkey-composed install --start

# Edit the config to declare your mounts
vim ~/.config/turnkey/composed.toml

With home-manager (declarative)

{
  imports = [ turnkey.homeManagerModules.turnkey-composed ];

  services.turnkey-composed = {
    enable = true;
    package = turnkey.packages.${system}.turnkey-composed;
    mounts = {
      myproject = {
        repo = "/Users/me/src/myproject";
        mountPoint = "/firefly/myproject";
      };
    };
  };
}

Prerequisites

Linux

# Verify FUSE is available
ls /dev/fuse

# If missing, install fuse3
sudo apt install fuse3  # Debian/Ubuntu
sudo dnf install fuse3  # Fedora

macOS

Install FUSE-T (no kernel extension, works on Apple Silicon):

brew install macos-fuse-t/homebrew-cask/fuse-t

Mount points under /: macOS root is read-only. The daemon automatically manages /etc/synthetic.conf entries and activates them via apfs.util -t when a mount point like /firefly/turnkey is requested. This requires sudo (the daemon prompts when needed).

For paths under ~ (e.g., ~/firefly/turnkey), no special setup is needed.

Service Configuration

The service reads ~/.config/turnkey/composed.toml:

# Mount a project
[[mounts]]
repo = "/Users/me/src/myproject"
mount_point = "/firefly/myproject"

# Mount another project
[[mounts]]
repo = "/Users/me/src/other-project"
mount_point = "/firefly/other"
backend = "fuse"  # Optional: "auto" (default), "fuse", or "symlink"

The daemon watches this file for changes. When you add a new [[mounts]] entry, the daemon picks it up and mounts it automatically — no restart needed.

Home-Manager Module

The declarative alternative to editing the TOML file directly:

{
  imports = [ turnkey.homeManagerModules.turnkey-composed ];

  services.turnkey-composed = {
    enable = true;
    package = turnkey.packages.${system}.turnkey-composed;
    mounts = {
      myproject = {
        repo = "/Users/me/src/myproject";
        mountPoint = "/firefly/myproject";
      };
      other = {
        repo = "/Users/me/src/other";
        mountPoint = "/firefly/other";
        backend = "fuse";  # Optional
      };
    };
  };
}

This generates the config file and manages the launchd agent (macOS) or systemd user service (Linux).

Service Management

# Install and start the service
turnkey-composed install --start

# Uninstall the service
turnkey-composed uninstall

# The service runs `turnkey-composed serve` which:
# - Reads ~/.config/turnkey/composed.toml
# - Builds cells via nix for each repo
# - Mounts all entries
# - Watches for config and manifest changes

A buck2 daemon keeps the mount it started on. When turnkey-composed mounts with FUSE, it kills the buck2 daemons of the projects inside the mount point, so a restarted service needs no buck2 kill: the next buck2 command starts a fresh daemon.

How Cell Discovery Works

On startup, turnkey-composed:

  1. Runs nix eval to list *-cell packages from each repo's flake
  2. Runs nix build to build all cells in a single invocation (~3-4s if cached)
  3. Uses the Nix store paths to populate external/ in the FUSE mount

Cells are always built from the current flake state. The daemon watches manifest files (go-deps.toml, rust-deps.toml, etc.) and rebuilds cells automatically when they change.

Mount Structure

/firefly/myproject/
├── .buckconfig             # Virtual - generated by layout
├── .buckroot               # Virtual - marks Buck2 root
├── root/                   # Pass-through to your repository
│   ├── src/
│   ├── docs/
│   ├── flake.nix
│   └── ...
└── external/               # Dependency cells (from Nix store)
    ├── godeps/
    ├── rustdeps/
    ├── prelude/
    ├── toolchains/
    └── ...

Buck2 runs from the mount root. Source targets use the root// cell prefix: buck2 build root//src/cmd/tk:tk.

CLI Reference

Single Mount

# Start (foreground)
turnkey-composed start --mount-point <path> --repo-root <path> [--backend fuse|symlink|auto]

# With explicit config file
turnkey-composed start --config <path>

Service Mode

# Run as a service (reads ~/.config/turnkey/composed.toml)
turnkey-composed serve [--config <path>]

# Install/uninstall the system service
turnkey-composed install [--start]
turnkey-composed uninstall

Control

turnkey-composed status     # Check daemon status
turnkey-composed refresh    # Trigger manual cell rebuild
turnkey-composed stop       # Stop the daemon

Platform Notes

Linux

Uses native FUSE via /dev/fuse with the fuser Rust crate. Best performance.

macOS

Uses FUSE-T with direct C FFI bindings to libfuse3. FUSE-T translates FUSE operations to NFS internally. No kernel extension required.

The daemon handles synthetic firmlinks automatically for mount points under / (manages /etc/synthetic.conf and runs apfs.util -t).

Fastest for CI. No daemon needed. Automatically selected when FUSE is unavailable.

Integration with IDEs

VS Code / Cursor

{
  "go.goroot": "/firefly/myproject/root",
  "rust-analyzer.linkedProjects": ["/firefly/myproject/root/Cargo.toml"]
}

IntelliJ / GoLand

Set the project root to the FUSE mount point for consistent path resolution.

Building Projects

Turnkey integrates with Buck2 for building projects.

The tk Command

Use tk instead of buck2 directly. It provides:

  • Automatic dependency sync before builds
  • A daemon restart when a symlinked cell changed (Symlinked Cells and Plain buck2)
  • Consistent behavior across the team
tk build //path/to:target

Common Build Commands

# Build a specific target
tk build //src/examples/go-hello:go-hello

# Build all targets
tk build //...

# Build with verbose output
tk build //... -v

# Build in release mode
tk build //... -c release

Build Outputs

Build outputs are placed in buck-out/.turnkey/:

buck-out/
└── .turnkey/
    ├── gen/
    │   └── root/
    │       └── path/to/target/
    └── tmp/
        └── ...

Why .turnkey? The isolation directory starts with a dot so that language tools ignore it:

  • Go skips directories starting with . when scanning for packages
  • Cargo ignores dot-directories
  • pytest ignores dot-directories by default

This prevents errors like Go trying to parse generated .go files in build outputs, or pytest collecting test files from there.

To find the output path for a specific target:

tk build //path/to:target --show-output

Skipping Sync

If you know dependencies haven't changed:

tk --no-sync build //...

Troubleshooting

Missing Toolchain

If you see "toolchain not found", ensure:

  1. The toolchain is declared in toolchain.toml
  2. You've re-entered the shell after adding it

Stale Dependencies

If builds fail with missing dependencies:

tk sync
tk build //...

Running Tests

Turnkey supports running tests via Buck2.

Test Commands

# Run tests for a specific target
tk test //path/to:target-test

# Run all tests
tk test //...

# Run tests matching a pattern
tk test //src/examples/...

tk test reuses the recorded result of a test whose inputs haven't changed instead of running it again; see Test Result Caching. Use tk --rerun test to run everything.

Language-Specific Tests

Go Tests

tk test //src/go/pkg/mypackage:mypackage_test

Rust Tests

tk test //src/rust/mycrate:mycrate-test

Python Tests

tk test //src/python/mymodule:test

Test Output

Only tests that didn't pass are listed, each with its stdout and stderr; the summary line counts the rest. To list passing tests with their output too:

tk test //... -- --print-passing-details

Filtering Tests

Arguments after -- go to the test runner, not the test binary. Pass them on to the binary with --test-arg, which takes every argument after it, so it comes last:

# Run specific test function (Go)
tk test //pkg:pkg_test -- --test-arg -test.run=TestSpecificFunction

# Run specific test (Rust)
tk test //crate:crate-test -- --test-arg specific_test_name

A filtered run has its own result key, so it doesn't reuse the unfiltered run's result.

Continuous Testing

For development, use Buck2's file watching:

tk test //path/to:target-test --watch

Test Result Caching

tk test reuses a test's recorded result when nothing it depends on has changed, instead of running it again. buck2 already caches build actions; test result caching extends that to test runs. It is on by default.

$ tk test //src/...
Tests finished: Pass 42. Fail 0. Timeout 0. Fatal 0. Skip 0. Omit 0. Infra Failure 0. Build failure 0
38 recorded (reused without running)

A reused result is a hit. The last line counts the hits. Exit codes are the same as when every test runs: 0 if all tests pass, 32 if any fails.

Passing tests, hits included, aren't listed: only tests that didn't pass are, with their output. To list every test with its output, pass --print-passing-details to the test runner:

$ tk test //src/... -- --print-passing-details
✓ Pass: root//src/rust/starlark-parse:starlark-parse-test
recorded: reused the result of an earlier run with the same inputs
---- STDOUT ----
...

A hit is then marked recorded under the test's line, shows no duration (the test didn't run), and prints the output of the run that recorded it.

When a result is reused

A result is reused only when its result key is unchanged. The key covers everything the test can see:

  • its inputs (sources, dependencies, data files, the toolchain);
  • its command line, including arguments after -- such as --test-arg (so a filtered run has its own key);
  • its declared environment, including --env;
  • its timeout, working directory and platform;
  • the buck2 release and the caching tool's version.

It works across buck2 daemon restarts, tk clean, and other checkouts of the same revision on the same machine (jj workspaces, git worktrees).

Only passes are recorded. A test that failed, timed out or crashed always runs again.

Caching applies only under tk test. A plain buck2 test neither reuses nor records results.

Forcing a re-run

tk --rerun test //src/...

runs every matched test, and records the fresh passes, which replace the old ones. Like --no-sync, the flag goes before the subcommand.

Which targets are cached

Every target of a cache-safe rule is: turnkey's rules and the rust, go and python test rules. With caching on, they carry the label turnkey-cacheable, which buck2 uquery shows. It is added by the rules, never by hand.

Opting a target out

Label a target no-test-cache to always run it and never record it, for example a test that talks to the network or depends on the time:

go_test(
    name = "integration_test",
    srcs = ["integration_test.go"],
    # Talks to a staging server.
    labels = ["no-test-cache"],
)

For solidity_test, the label also switches fuzzing back to a random seed; cached Solidity tests fuzz with a seed derived from the target's label. A solidity_test with fork_url is never cached, because it reads chain state over the network.

What each language does

A test is only cached when its rule can't read anything outside its result key. For cached tests, turnkey:

  • sets PATH to Nix store paths only (bash, coreutils, diffutils), and HOME to /homeless-shelter, a directory that doesn't exist. A test that needs a writable directory should use $TMPDIR;
  • runs the command with project-relative paths, from the project root.
LanguageNotes
RustNo other changes.
GoTests that read fixtures must declare them as resources.
PythonRuns the toolchain's interpreter by store path, with PYTHONDONTWRITEBYTECODE=1.
Jsonnetimport only resolves from the test's declared sources and dependencies.
Solidityforge uses the toolchain's solc, offline, with dependencies from the soldeps cell.

Configuration

Test result caching is configured in your flake, under the Buck2 integration:

turnkey.toolchains.buck2.testCache = {
  enable = true;       # default; false runs tests under buck2's bundled runner
  endpoint = null;     # default: a cache on this machine, managed by tk
  tls = true;          # only used with a remote endpoint
};

The local cache

Recorded results live in one store per user per machine, shared by every checkout and every turnkey repo:

PlatformLocation
macOS~/Library/Caches/turnkey/test-results/
Linux~/.cache/turnkey/test-results/

tk test starts the cache server (bazel-remote) on demand, in the background. It stops by itself after 24 hours without use, and the next tk test starts it again; recorded results stay in the store. Its log is server.log in the store. The store is limited to 5 GiB; the least recently used results are dropped first.

If the cache can't be reached or started, tk test runs the tests uncached and prints one line:

tk: running tests without the test result cache: <reason>

Per-user settings

These are set in your environment, not in the repo:

VariableEffect
TURNKEY_CACHE_DIRKeep turnkey's caches here instead of the platform cache directory.
TURNKEY_TEST_CACHE_SIZE_GIBStore size limit in GiB (default 5).
TURNKEY_TEST_CACHE_PORTPort of the local cache server (default 47301). Read when the dev shell is evaluated, so reload the shell after changing it.

A remote cache

Set endpoint to use a shared Remote Execution API cache instead of the local one:

turnkey.toolchains.buck2.testCache.endpoint = "grpc://cache.example.com:443";

tk test then starts no local cache, and results found there are marked recorded, remote. Nothing is recorded to a remote cache yet: who may write to a shared cache hasn't been decided. tk can't check a remote cache before running, so if it is unreachable, buck2 retries for about 45 seconds and then runs the tests locally, uncached.

In the event log

For every hit, the test result in buck2's event log (buck2 log show) carries a JSON record in its msg field:

{"turnkey_test_cache": {"hit": true, "origin": "local", "original_duration_us": 475340}}

origin is local or remote, and original_duration_us is how long the recorded run took. The test's TestEnd event reports the execution as a RemoteCommand with cache_hit: true.

Caching your own test rules

A custom test rule opts in by passing the arguments it gives ExternalRunnerTestInfo through test_caching_kwargs:

load("@prelude//test_caching:test_caching.bzl", "test_caching_kwargs")

def _my_test_impl(ctx):
    command = cmd_args(ctx.attrs.runner[RunInfo], ctx.attrs.src)
    return [
        DefaultInfo(),
        ExternalRunnerTestInfo(**test_caching_kwargs(
            {
                "type": "my_language",
                "command": [command],
                "env": ctx.attrs.env,
                "labels": ctx.attrs.labels,
            },
            # Store-path bin directories the test needs beyond bash,
            # coreutils and diffutils.
            extra_path = [],
            # Variables only cached runs need.
            extra_env = {},
        )),
    ]

When caching is off, the arguments come back unchanged. When it is on, the helper also gives the test an executor that reads recorded results. A rule that supports remote execution passes its re_executors as well, and a test that upstream runs remotely keeps its executor. Only opt a rule in when its tests can't read anything that isn't in their result key: every file they read must be a declared input, and every tool must come from a store path.

Managing Dependencies

This guide covers how external dependencies are managed in Turnkey projects.

Core Principles

1. No In-Repo Vendoring

Dependencies are never vendored into the repository. All dependency sources live in the Nix store.

  • No vendor/ directories committed to git
  • No node_modules/, __pycache__/, or similar cached dependencies
  • The repository contains only source code and dependency declarations

2. Language-Native Declarations Are the Source of Truth

Each language has its own dependency declaration format. These are the sole source of truth for what dependencies are needed:

LanguageDeclaration Files
Gogo.mod, go.sum
RustCargo.toml, Cargo.lock
Pythonpyproject.toml, uv.lock

These files define the dependency graph at the module level (not package/subpackage level).

3. Per-Module Fetching with Deterministic Hashes

Dependencies are fetched individually by Nix, each with its own content hash:

go.mod/go.sum  →  godeps-gen  →  go-deps.toml  →  Nix fetches each module

The intermediate TOML file (go-deps.toml, rust-deps.toml, etc.) contains:

  • Module/crate/package identifiers
  • Versions (from lock file)
  • Nix-compatible SRI hashes (from prefetching)

4. Dependency Cells for Buck2

Dependencies are assembled into Buck2 cells by Nix:

go-deps.toml  →  one Nix package per module  →  .turnkey/godeps/  (tk materialize)

The cell contains:

  • Fetched source files for each dependency
  • Generated rules.star files for Buck2 to consume
  • Any scaffolding needed by build tools (e.g., modules.txt for Go)

Data Flow

┌─────────────────────────────────────────────────────────────────────────┐
│                          Source of Truth                                │
│                                                                         │
│   go.mod / go.sum          Cargo.toml / Cargo.lock       pyproject.toml │
└─────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Hash Generation Tools                            │
│                                                                         │
│   godeps-gen                      rustdeps-gen             pydeps-gen   │
│                                                                         │
│   Reads dependency declaration, fetches each module via nix-prefetch-*  │
│   Outputs TOML with per-module SRI hashes                               │
└─────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Dependency TOML Files                            │
│                                                                         │
│   go-deps.toml                rust-deps.toml           python-deps.toml │
│                                                                         │
│   [deps."github.com/foo/bar"]                                           │
│   version = "v1.2.3"                                                    │
│   hash = "sha256-..."                                                   │
└─────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Nix Cell Builders                                │
│                                                                         │
│   nix/lib/deps-cell/adapters/{go,rust,python,javascript,solidity}.nix   │
│                                                                         │
│   - Reads TOML, fetches each module via fetchFromGitHub/fetchurl        │
│   - Assembles into directory structure                                  │
│   - Generates rules.star files                                          │
└─────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                        Buck2 Cells (in .turnkey/)                       │
│                                                                         │
│   .turnkey/godeps/           .turnkey/rustdeps/       .turnkey/pydeps/  │
│   (directories of links into the Nix store)                             │
│                                                                         │
│   Contains: source files, rules.star files, cell config                 │
└─────────────────────────────────────────────────────────────────────────┘
                                      │
                                      ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                             Buck2 Build                                 │
│                                                                         │
│   buck2 build //my/package:target                                       │
│                                                                         │
│   References deps as: godeps//vendor/github.com/foo/bar:bar             │
│   All sources already in Nix store - no network access needed           │
└─────────────────────────────────────────────────────────────────────────┘

Auto-Sync with Wrapped Tools

When using go, cargo, or uv in a Turnkey shell, the tools are transparently wrapped to trigger automatic dependency synchronization when dependency files change.

# These trigger auto-sync when dependency files change
go get github.com/some/package
cargo add serde
uv add requests

How Auto-Sync Works

  1. The wrapper captures a hash of dependency files before running the command
  2. The actual tool runs (e.g., go get)
  3. After completion, the wrapper checks if dependency files changed
  4. If changed, tk sync is triggered automatically

Verbose Mode

Use verbose mode to see what the wrapper is doing:

tw -v go get github.com/some/package

Manual Sync

Force a full dependency sync with:

tk sync

Or sync specific languages:

tk sync --go
tk sync --rust
tk sync --python

Go Dependencies

Configuration

turnkey.toolchains.buck2.go = {
  enable = true;
  depsFile = ./go-deps.toml;
};

Generating go-deps.toml

tk sync regenerates it when go.mod or go.sum changes. To run the generator yourself:

godeps-gen -o go-deps.toml

Options:

  • --no-prefetch: Skip fetching the Nix hashes of the modules' proxy.golang.org zips, the source the godeps cell fetches from (the hashes are then invalid)
  • --no-cache: Always fetch from the network, bypassing the prefetch cache
  • --indirect: Include indirect (transitive) dependencies (default: true)
  • -o, --output: Output file (default: stdout)

Using Dependencies in Build Files

go_binary(
    name = "hello",
    srcs = ["main.go"],
    deps = [
        "godeps//vendor/github.com/spf13/cobra:cobra",
    ],
)

Multiple Modules and Local Replaces

Several Go modules in one repo are declared as a go.work workspace at the project root. Go resolves them together into one go-deps.toml and one godeps cell, and rules sync maps imports of a member to its targets in the repo. A local-path replace directive must point at a workspace member.

See the Go language guide for detailed documentation.

External Fork Replace Directives

Turnkey also supports replace directives that point to external forks:

In go.mod:

replace github.com/original/pkg => github.com/myfork/pkg v1.2.3

In go-deps.toml (generated by godeps-gen):

[deps."github.com/original/pkg@v1.2.3"]
import_path = "github.com/original/pkg"
fetch_path = "github.com/myfork/pkg"
version = "v1.2.3"
hash = "sha256-..."

The cell builder fetches from fetch_path but stores under import_path, so your code continues importing from the original path while using the fork's source.

See the Go language guide for detailed documentation.

Rust Dependencies

Configuration

turnkey.toolchains.buck2.rust = {
  enable = true;
  depsFile = ./rust-deps.toml;
};

Generating rust-deps.toml

rustdeps-gen --cargo-lock Cargo.lock -o rust-deps.toml

Options:

  • --cargo-lock: Path to Cargo.lock file (default: Cargo.lock)
  • --no-prefetch: Skip prefetching (produces incorrect hashes)
  • --no-cache: Always fetch from the network, bypassing the prefetch cache
  • -o, --output: Output file (default: stdout)

Handling Special Cases

Some Rust crates require additional configuration. See the Rust Dependency Handling guide for:

  • Build scripts that emit rustc flags
  • Generated source files
  • Native code compilation

Python Dependencies

Configuration

turnkey.toolchains.buck2.python = {
  enable = true;
  depsFile = ./python-deps.toml;
};
# 1. Generate lock file from pyproject.toml
uv lock

# 2. Export to PEP 751 format
uv export --format pylock.toml -o pylock.toml

# 3. Generate python-deps.toml with Nix hashes
pydeps-gen --lock pylock.toml -o python-deps.toml

Input Formats

FormatFlagReproducibilityNotes
pylock.toml (PEP 751)--lockBestExact versions and URLs
pyproject.toml--pyprojectVariesUses latest matching versions
requirements.txt--requirementsVariesPin versions with == for reproducibility

CLI Options

--lock <PATH>          Path to pylock.toml (PEP 751 lock file) - RECOMMENDED
--pyproject <PATH>     Path to pyproject.toml
--requirements <PATH>  Path to requirements.txt
-o, --output <PATH>    Output file (default: stdout)
--no-prefetch          Skip prefetching (produces placeholder hashes)
--no-cache             Always fetch from the network, bypassing the prefetch cache
--include-dev          Include dev dependencies from optional-dependencies.dev

Anti-Patterns to Avoid

Never Use vendorHash

Nix's buildGoModule has a vendorHash that hashes the output of go mod vendor. This is problematic:

  1. Implementation-dependent: The hash changes based on which packages are actually imported
  2. Opaque: You can't know the hash without running the build and letting it fail
  3. Unstable: Adding a new import from an existing module can change the hash

Instead, use per-module fetching where each module has its own deterministic hash.

Never Vendor in Repository

Even temporarily. If you see a vendor/ directory in the repo, something is wrong.

Never Compute Hashes from Vendored Output

The hash should come from the source (e.g., GitHub tarball), not from transformed/vendored output.

The Go, Rust, Python, Solidity and JavaScript Cells Are Real Directories

.turnkey/godeps, .turnkey/rustdeps, .turnkey/pydeps, .turnkey/soldeps and .turnkey/jsdeps are directories that tk materialize keeps in line with the cell index the shell builds (ADR 0004).

  • _store/<store path name>: one symlink per crate, Go module, Python distribution or Solidity package, to its own store path. It is only ever created or deleted, never pointed elsewhere.
  • vendor/<crate>@<version>/rules.star and vendor/<crate>/rules.star: alias() targets that forward to a store link's crate. Labels such as rustdeps//vendor/anyhow:anyhow are unchanged.
  • vendor/<name>/rules.star: one per Python distribution, forwarding to its store link (ADR 0010). Labels such as pydeps//vendor/six:six are unchanged.
  • vendor/<import path>/rules.star: one per Go package, forwarding to its package in its module's store link (ADR 0008). Labels such as godeps//vendor/golang.org/x/sys/unix:unix are unchanged. Where a dependency imports a go.work member's package, its alias forwards to the member's target in the repo instead.
  • vendor/<name>@<version>/rules.star: one per npm package, forwarding to its files in its store link. The cell's root rules.star, which the index carries, declares the package graph over them: an instance per pnpm snapshot, and jsdeps//:<npm name> per direct dependency (ADR 0012).
  • vendor/<name>/rules.star: one per Solidity package, forwarding to its store link, beside links to the package's files, which native forge reads through the root remappings.txt (ADR 0011). Labels are unchanged: soldeps//:<package> and soldeps//:bundle come from the cell's root rules.star, which the index carries, and soldeps//vendor/<name>:<package> from the alias package.

Only what a change touches is rewritten, so a dependency bump recompiles the bumped crate's, module's or distribution's dependents and re-runs only their tests. An npm package bump re-runs the instances that depend on it and their consumers. A Solidity package bump re-runs every Solidity action, since each one stages the whole bundle, and nothing in another language. Everything else stays cached, with no daemon restart, and plain buck2 reads the new version. Don't edit the directory: the shell rewrites it on every load.

  • Switching over: the first shell load after upgrading turnkey replaces the old .turnkey/<cell> symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one.
  • Going back to an older turnkey: run rm -rf .turnkey/<cell>, then reload the shell.
  • tk: warning: the rustdeps cell was built from another rust-deps.toml (or godeps and go-deps.toml, pydeps and python-deps.toml, soldeps and solidity-deps.toml, jsdeps and js-deps.toml): the deps file changed since the shell last loaded. Run direnv reload, or re-enter the shell, to rebuild the cell.

Troubleshooting

Dependencies Not Found

If Buck2 can't find a dependency:

  1. Check that the deps TOML file is up to date:

    tk sync
    
  2. Verify the cell exists:

    ls -la .turnkey/godeps
    
  3. Check the target path format:

    # Correct format
    godeps//vendor/github.com/spf13/cobra:cobra
    
    # Wrong - missing vendor/ prefix
    godeps//github.com/spf13/cobra:cobra
    

Hash Mismatch Errors

If you get hash mismatch errors when building:

  1. Regenerate the deps file with fresh hashes:

    godeps-gen --no-cache -o go-deps.toml
    
  2. Re-enter the dev shell:

    exit
    nix develop
    

Stale Dependencies

If dependency changes aren't picked up:

  1. Kill the Buck2 daemon, if you build with plain buck2:

    buck2 kill
    

    A running daemon keeps what it read through a repointed symlink (Symlinked Cells and Plain buck2); tk restarts it itself.

  2. Force a full sync:

    tk sync
    

Dependency Fixups

Some dependencies don't build under Buck2 as they are. A Rust crate's build.rs never runs, so whatever it generates, compiles or tells rustc has to come from somewhere else; a dependency may need a patch. A fixup is what turnkey supplies for one dependency to make it build: a patch, the output its build script would generate, the flags it would pass.

Fixups come in fixup sets: modules of class turnkeyFixups (ADR 0003). Your repository brings the sets it needs, and its own fixups, through one option. turnkey applies no fixups you didn't bring.

Bringing fixups

perSystem = { ... }: {
  turnkey.toolchains.buck2.fixups = {
    # Sets published by other flakes
    imports = [
      inputs.turnkey.modules.turnkeyFixups.serde
      inputs.acme-fixups.modules.turnkeyFixups.default
    ];

    # This repository's own fixups
    rust.zerocopy.buildScript.skip = true;
  };
};

A fixup is keyed by the dependency's name in its own ecosystem: rust.<crate>, go."<import path>", python.<distribution>, javascript.<package>, solidity.<package>.

turnkey's published fixups

turnkey publishes the fixups its own repository needs, one module per family, under inputs.turnkey.modules.turnkeyFixups:

ModuleCrates
serdeserde, serde_core, serde_json
thiserrorthiserror
ringring 0.17
rustixrustix
nixnix
fuserfuser
tree-sittertree-sitter and its rust, python, solidity, starlark and typescript grammars
build-script-skipscrates whose build script turnkey's builds need nothing from
defaultall of the above

Import only what your lock needs: an imported fixup for a crate you don't lock is silently unused.

Every build script needs a fixup

Buck2 never runs build.rs, so every crate you lock that has one needs a fixup saying what stands in for it. The Rust cell fails to build otherwise:

error: turnkey: serde 1.0.228 has a build.rs, and no fixup says what stands in for it; a published fixup set does: add `inputs.turnkey.modules.turnkeyFixups.serde` to turnkey.toolchains.buck2.fixups.imports

When no published set accounts for the crate, the error says how to write the fixup. If the crate's build script only probes the compiler or the target, or emits cfgs for features you don't use, the build needs nothing from it:

rust.zerocopy.buildScript.skip = true;

Otherwise, give it a build script generating what its build.rs would, and the flags it would pass. The developer manual's Dependency Generators page describes the whole record and how to diagnose what a crate needs.

Versions

A fixup applies to every locked version of its dependency. Fields that hold only for some versions go in versions entries, whose bounds are compared with the locked version:

rust.ring.versions = [
  {
    when = { atLeast = "0.17"; below = "0.18"; };
    buildScript.generate = ...;
  }
];

Every entry whose bounds hold applies. A version no entry covers gets only the fixup's other fields, so a new major version of a crate is reported as unaccounted for rather than built with a fixup written for another.

Patches

rust.some-crate.patches = [ ./patches/some-crate-fix.patch ];
go."github.com/foo/bar".patches = [ ./patches/bar.patch ];

A fixup's patches apply to the dependency's own source, in order, with -p1: a plain git diff in a checkout of the dependency works as is. Patches from several sets apply in import order, before a Rust crate's build script runs.

Fixup patches are separate from the patches tk compose patch writes to .turnkey/patches/<cell>/ from the FUSE edit layer. Those are this repository's local, exact-version workarounds.

  • Rust cell: each patch goes in its package's directory, .turnkey/patches/rustdeps/vendor/<crate>@<version>/, and applies in that crate's own derivation, after its fixup. Changing a patch rebuilds only that crate and what depends on it.
    • A directory may also be named after the crate alone (vendor/anyhow/); it then goes to the version the cell's unversioned alias points at.
    • A patch that doesn't apply exactly, with no fuzz, fails the build and names the crate.
    • A patch file left directly under rustdeps/, from before this layout, fails evaluation: move it into its package's directory, or regenerate it with tk compose patch.
  • Go cell: each patch goes in its module's directory, .turnkey/patches/godeps/vendor/<module path>/, and applies in that module's own derivation, after its fixup.
  • Python cell: each patch goes in its distribution's directory, .turnkey/patches/pydeps/vendor/<name>/, and applies in that distribution's own derivation, after its fixup and before its rules.star is written. Changing a patch rebuilds only that distribution and what depends on it.
    • Patches apply to the distribution's unpacked wheel, the installed layout, not its sdist. A patch written against sdist paths (such as vendor/requests/src/requests/...) doesn't apply: regenerate it with tk compose patch.
    • A patch that doesn't apply exactly, with no fuzz, fails the build and names the distribution.
    • A patch file left directly under pydeps/, from before this layout, fails evaluation: move it into its distribution's directory, or regenerate it with tk compose patch. So does a directory naming a distribution python-deps.toml doesn't hold.
  • Solidity cell: each patch goes in its package's directory, .turnkey/patches/soldeps/vendor/<name>/ (for a scoped npm package, vendor/@<scope>/<name>/, such as vendor/@openzeppelin/contracts/), and applies in that package's own derivation, after its fixup. Changing a patch rebuilds only that package.
    • A patch that doesn't apply exactly, with no fuzz, fails the build and names the package.
    • A patch file left directly under soldeps/, from before this layout, or in a directory that is no package's, fails evaluation: move it into its package's directory, or regenerate it with tk compose patch.
  • JavaScript cell: each patch goes in its package's directory, .turnkey/patches/jsdeps/vendor/<name>@<version>/ (for a scoped package, vendor/@<scope>/<name>@<version>/), and applies in that package's own derivation, after its fixup, so to every instance of it. Changing a patch rebuilds only that package.
    • A directory may also be named after a direct dependency alone (vendor/lodash/); it then goes to the version the root package.json resolves it to.
    • A patch that doesn't apply exactly, with no fuzz, fails the build and names the package.
    • A patch file left directly under jsdeps/, from before this layout, or in a directory that is no package's, fails evaluation: move it into its package's directory, or regenerate it with tk compose patch.

When sets disagree

Fixups merge field by field, as NixOS modules do: flags and patches from every set concatenate. Two sets that give a crate different build scripts, or an environment variable different values, fail evaluation, naming both files:

error: The option `rust.serde.buildScript.generate.<function body>' has conflicting definition values:
- In `conflicting/flake.nix#modules.turnkeyFixups.default': "echo another serde"
- In `acme/flake.nix#modules.turnkeyFixups.default': "..."

Resolve it in your own fixups:

  • Override one field with lib.mkForce: rust.serde.buildScript.generate = lib.mkForce "...";
  • Drop one fixup an imported set brings: rust.ring.enable = false;
  • Drop a whole module: disabledModules = [ ... ];, or don't import it.

Unused fixups

A fixup written in your own turnkey.toolchains.buck2.fixups that matches no locked dependency, or a versions entry matching none of its locked versions, warns at evaluation: it usually means a rename or an upgrade left it behind. Fixups from imported sets never warn, so one organization-wide set can serve many repositories that each lock only part of it.

Publishing a fixup set

A fixup set is a plain module; publish it from any flake as modules.turnkeyFixups.<name>. With flake-parts, import its modules module, which stamps the module's class:

{
  imports = [ inputs.flake-parts.flakeModules.modules ];

  flake.modules.turnkeyFixups = {
    openssl = ./fixups/openssl.nix;
    default = { imports = [ ./fixups/openssl.nix ]; };
  };
}

Without flake-parts, set the class yourself:

outputs = { self, ... }: {
  modules.turnkeyFixups.default = {
    _class = "turnkeyFixups";
    imports = [ ./fixups/openssl.nix ];
  };
};

A set's module receives pkgs and lib, and may hold fixups for several languages at once, such as for a library packaged for both Rust and Python:

# fixups/acme-proto.nix
{ ... }:
{
  rust.acme-proto = {
    buildScript.skip = true;
    patches = [ ./acme-proto-rust.patch ];
  };
  python.acme-proto.patches = [ ./acme-proto-python.patch ];
}

Python Workspaces

Turnkey lays out Python source as a uv workspace, in parallel to the Cargo workspace pattern used for Rust. A single uv.lock resolves every Python package in the monorepo against a consistent dependency set, while each package keeps its own pyproject.toml declaring exactly what it consumes.

Two tracks run side by side over the same source:

  • uv track — uv sync, uv run, IDE language servers, REPL. Members are installed editable so source edits are reflected immediately.
  • Buck2 track — tk build, tk test. External packages are vendored into the pydeps cell built from python-deps.toml.

Repository Layout

/repo/
├── pyproject.toml                       # Workspace root: members + uv.lock anchor
├── uv.lock                              # Single resolved lockfile (managed by uv)
├── pylock.toml                          # PEP 751 export from uv.lock
├── python-deps.toml                     # Generated for Buck2/Nix from pylock.toml
└── src/python/<member>/
    ├── pyproject.toml                   # [project] + hatchling build backend
    ├── rules.star                       # Buck2 targets for the member
    └── turnkey/<member>/                # Source under shared turnkey.* namespace
        ├── __init__.py
        └── ...

Tests live in a sibling tests/ directory inside each member, kept outside the importable namespace.

The turnkey.* Namespace Convention

Every workspace member contributes a subpackage under the shared turnkey PEP 420 implicit namespace package. No member defines a top-level turnkey/__init__.py; Python's import system resolves turnkey.parser, turnkey.config, etc. by walking every sys.path entry that exposes a turnkey/<name>/ directory.

Downstream Projects: Pick Your Own Namespace

The turnkey.* prefix is this repository's namespace. If you adopt the same workspace pattern in a different monorepo, choose a namespace specific to your organisation — e.g. acme.<name> — to avoid colliding with packages on PyPI or other turnkey-based repos. The mechanics are identical; substitute turnkey for your namespace throughout this guide.

Member pyproject.toml

Each library member uses the hatchling backend and points it at the turnkey/ directory:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "turnkey-parser"
version = "0.1.0"
description = "Cargo manifest and feature-graph utilities"
requires-python = ">=3.11"
dependencies = [
    "turnkey-config",       # cross-member dep
]

[tool.uv.sources]
turnkey-config = { workspace = true }

[tool.hatch.build.targets.wheel]
packages = ["turnkey"]   # everything under turnkey/<name>/ is the wheel content

packages = ["turnkey"] is the key line: it tells hatchling that the wheel's content is whatever lives under the turnkey/ directory of this member. Combined with PEP 420 namespace resolution, every member ships only its own turnkey/<name>/ slice without anyone owning turnkey/__init__.py.

Cross-Member Dependencies

Declare the dep under [project] dependencies with the bare package name, then pin its source to the workspace under [tool.uv.sources]:

dependencies = ["turnkey-config"]

[tool.uv.sources]
turnkey-config = { workspace = true }

This mirrors Cargo.toml's serde.workspace = true pattern — the consumer member doesn't pin a version, the lockfile reconciles it.

External Dependencies

Declare externals in the member that consumes them, never the workspace root:

# src/examples/python-hello-deps/pyproject.toml
[project]
name = "turnkey-example-python-hello-deps"
dependencies = ["six>=1.16.0"]

The single uv.lock at the workspace root resolves every external version-consistently across members.

Non-Packaged Members

Some members exist only to declare dependencies, not to be installed (typical for application-like entrypoints or examples). Mark them non-packaged:

[project]
name = "turnkey-example-python-hello-deps"
version = "0.1.0"
dependencies = ["six>=1.16.0"]

[tool.uv]
package = false        # uv won't build/install this member

No [build-system] is required. uv still resolves the member's dependencies as part of the workspace lock.

Root pyproject.toml

The workspace root anchors membership and the shared lockfile:

[project]
name = "turnkey"
version = "0.1.0"
requires-python = ">=3.11"

# Listing members as dependencies makes the default `uv sync` install all
# of them in one shot — no `--all-packages` flag needed.
dependencies = [
    "turnkey-parser",
    "turnkey-config",
    "turnkey-example-python-hello",
    "turnkey-example-python-hello-deps",
]

[dependency-groups]
# Dev tooling — auto-installed by 'uv sync' so 'uv run pytest' Just Works.
dev = ["pytest>=7.0"]

[tool.uv.workspace]
members = [
    "src/python/parser",
    "src/python/config",
    "src/examples/python-hello",
    "src/examples/python-hello-deps",
]

[tool.uv.sources]
turnkey-parser = { workspace = true }
turnkey-config = { workspace = true }
turnkey-example-python-hello = { workspace = true }
turnkey-example-python-hello-deps = { workspace = true }

[tool.uv]
package = false        # the root itself isn't a packaged project

Buck2 Integration

Member source paths are spelled relative to the member's rules.star:

load("@prelude//:rules.bzl", "python_library", "python_test")

python_library(
    name = "parser",
    srcs = [
        "turnkey/parser/__init__.py",
        "turnkey/parser/grammar.py",
        "turnkey/parser/tokens.py",
    ],
    base_module = "",
    deps = ["//src/python/config:config"],
    visibility = ["PUBLIC"],
)

python_test(
    name = "test_parser",
    srcs = ["tests/test_parser.py"],
    base_module = "tests",
    deps = [":parser"],
)

base_module = "" tells Buck2 to install sources at their declared srcs paths, so files land at turnkey/parser/... in the runtime tree — matching the import prefix the rest of the codebase uses.

Adding or Updating Dependencies

# 1. Edit the member that needs the dep
$EDITOR src/python/parser/pyproject.toml      # add to [project] dependencies

# 2. Refresh editable installs (optional but recommended)
uv sync

# 3. Refresh the Buck2 pipeline
tk sync

uv add and uv remove do the same in one step: the uv wrapper runs tk sync itself when they change uv.lock.

tk sync runs two rules, in order:

  1. pylock re-exports pylock.toml from uv.lock when uv.lock or pyproject.toml is newer, relocking first if pyproject.toml changed: uv export --all-packages --no-dev --format pylock.toml. --all-packages includes externals from every member, and --no-dev keeps dev tooling (pytest etc.) out of the pydeps cell.
  2. python regenerates python-deps.toml from pylock.toml, and from uv.lock the dependency graph: each dependency's environment marker and each package's extras.

The pylock rule exists when the flake sets buck2.python.uvLockFile (turnkey's own flake sets it to uv.lock) along with buck2.python.lockFile.

One Version per Distribution

The pydeps cell holds one version of each distribution, at pydeps//vendor/<name>:<name> (ADR 0010). uv can lock several: when the resolution forks on a marker, for example numpy 1.x for python_version < '3.10' and 2.x above, the lock holds one entry per fork. pydeps-gen then fails, writes nothing, and names the distribution with each locked version and its marker:

pylock.toml locks several versions of one distribution, and the pydeps cell holds one version per distribution.
2 versions of numpy:
  numpy 1.26.4 (python_full_version < '3.10')
  numpy 2.1.0 (python_full_version >= '3.10')
Pin it, in the pyproject.toml that depends on it, to a range one version satisfies on every Python the workspace allows, so the lock no longer forks; then run tk sync.

Pin the dependency so the lock no longer forks: constrain it, in the pyproject.toml of the member that depends on it, to a range one version satisfies for every Python the workspace allows (numpy>=2.1), or raise the workspace's requires-python so the fork's other branch can't happen. Then run tk sync, which relocks and regenerates python-deps.toml.

Running Code

Taskuv trackBuck2 track
Run all testsuv run pytesttk test //src/python/...
Run a single member's testsuv run pytest src/python/parsertk test //src/python/parser:test_parser
Run an exampleuv run --package <pkg-name> <script>tk run //src/examples/python-hello-deps:python-hello-deps
REPL with members availableuv run pythonn/a
IDE language serverPoint at .venv/bin/pythonn/a

Both tracks resolve external dependencies the same way (uv.lock is the single source of truth), but the install paths differ: the uv track installs into .venv/, the Buck2 track materialises external packages into .turnkey/pydeps/vendor/<name>/.

See Also

Go Support

Turnkey provides comprehensive Go support with Buck2 integration.

Setup

Add to toolchain.toml:

[toolchains]
go = {}
godeps-gen = {}

Enable Go dependencies in flake.nix:

turnkey.toolchains.buck2.go = {
  enable = true;
  depsFile = ./go-deps.toml;
};

Project Structure

my-project/
├── go.mod
├── go.sum
├── go-deps.toml          # Generated from go.mod
├── cmd/
│   └── myapp/
│       ├── main.go
│       └── rules.star
└── pkg/
    └── mylib/
        ├── lib.go
        └── rules.star

Build Rules

In rules.star:

load("@prelude//go:go.bzl", "go_binary", "go_library")

go_binary(
    name = "myapp",
    srcs = ["main.go"],
    deps = ["//pkg/mylib:mylib"],
)

Tests

A go_test compiles its srcs with the package of its target_under_test, as go test does:

go_test(
    name = "mylib_test",
    srcs = glob(["*_test.go"]),
    target_under_test = ":mylib",
)

Test files can be in the package itself (package mylib) or in an external test package (package mylib_test), which sees only the package's exported API. Both kinds can be in the same go_test.

External Dependencies

Reference third-party packages via the godeps cell:

go_library(
    name = "mylib",
    srcs = ["lib.go"],
    deps = ["godeps//vendor/github.com/pkg/errors:errors"],
)

Auto-Sync

The go command is wrapped to auto-sync dependencies:

go get github.com/some/package  # Triggers sync
go mod tidy                      # Triggers sync

Multiple Modules: go.work

A repo with several Go modules declares them as a go.work workspace at the project root. This is the only supported multi-module shape (ADR 0007). Go resolves all the members together, so each third-party module has one version across the repo and one godeps cell. A repo without a go.work is a workspace of one member: the root go.mod.

my-monorepo/
├── go.work                   # use ./app ./shared-lib ./third_party/cobra-fork
├── go-deps.toml              # Generated by tk sync, lists the members
├── app/
│   ├── go.mod                # module github.com/company/app
│   └── cmd/server/
│       ├── main.go           # imports github.com/company/shared-lib/log
│       └── rules.star
├── shared-lib/
│   ├── go.mod                # module github.com/company/shared-lib
│   └── log/
│       ├── log.go
│       └── rules.star
└── third_party/cobra-fork/   # a patched local fork of github.com/spf13/cobra
    ├── go.mod                # module github.com/spf13/cobra
    └── rules.star

How imports map

The members are the only first-party Go code. tk sync records them in go-deps.toml under [members], and rules sync maps each import to the member whose module path is its longest prefix on a / boundary:

ImportTarget
github.com/company/shared-lib/log//shared-lib/log:log
github.com/spf13/cobra//third_party/cobra-fork:cobra
github.com/spf13/cobra/doc//third_party/cobra-fork/doc:doc
github.com/company/shared-libxnot a member: godeps//vendor/...

A package's target is named after its import path's last component, as in the godeps cell. That matters at a member's root: the root package of github.com/spf13/cobra in third_party/cobra-fork is :cobra, not :cobra-fork after its directory.

Local forks and replace

A member can be a local fork of a third-party module: give it the upstream module path and add it to go.work. Go then builds every import of that module from the fork, including imports from other third-party modules. In the godeps cell, those imports reach the member's targets through forwarding alias packages (ADR 0008).

A local-path replace directive, in go.work or a member's go.mod, must point at a workspace member. tk sync fails on one that points anywhere else: add the directory to go.work.

Modules outside the workspace

A go.mod that isn't a member is outside the Go build, as it is for go build ./... in the workspace. Rules sync leaves the Go rules under it alone. Test fixtures with their own go.mod need no fencing.

External Fork Replacements

Turnkey supports replace directives that point to external forks (not local paths). This is useful when:

  • Using a forked version of a dependency with bug fixes
  • Using a maintained fork of an abandoned project
  • Testing changes before upstreaming

How It Works

When godeps-gen encounters an external replace directive like:

replace github.com/original/pkg => github.com/myfork/pkg v1.2.3

It will:

  1. Set the dependency's import_path to github.com/original/pkg (for correct imports)
  2. Set the fetch_path to github.com/myfork/pkg (where to actually fetch from)
  3. Use the replacement version

In go.mod

module github.com/company/myapp

require github.com/original/pkg v1.0.0

replace github.com/original/pkg => github.com/myfork/pkg v1.2.3

Generated go-deps.toml

[deps."github.com/original/pkg@v1.2.3"]
import_path = "github.com/original/pkg"
fetch_path = "github.com/myfork/pkg"
version = "v1.2.3"
hash = "sha256-..."

How the Cell Builder Uses This

The Nix cell builder:

  1. Fetches the source from fetch_path (the fork)
  2. Stores it in the vendor directory under import_path (the original path)
  3. Generates Buck2 rules using the original import path

This means your code continues to import from the original path (github.com/original/pkg), but the actual source comes from your fork.

Version Handling

External replaces can change the version:

go.mod replaceResult
=> github.com/fork v1.2.3Uses v1.2.3 from fork
=> github.com/fork (no version)Uses the required version from fork

Version-specific replaces are also supported:

// Only replace v1.0.0, not other versions
replace github.com/pkg v1.0.0 => github.com/fork/pkg v1.0.1

Common Use Cases

Using a fork with a fix:

// Your fork has a critical bug fix not yet merged upstream
replace github.com/upstream/logger => github.com/you/logger v1.0.1-patched

Using a maintained fork:

// Original project abandoned, using community fork
replace github.com/old/abandoned => github.com/community/maintained v2.0.0

Testing before upstreaming:

// Test your changes before creating a PR
replace github.com/original/pkg => github.com/you/pkg v0.0.0-20240101

Rust Support

Turnkey provides Rust support with automatic dependency management.

Setup

Add to toolchain.toml:

[toolchains]
rust = {}
cargo = {}
rustdeps-gen = {}

Enable Rust dependencies in flake.nix:

turnkey.toolchains.buck2.rust = {
  enable = true;
  depsFile = ./rust-deps.toml;
};

Project Structure

my-project/
├── Cargo.toml
├── Cargo.lock
├── rust-deps.toml        # Generated by tk sync, from cargo
└── rust/
    └── mycrate/
        ├── src/
        │   └── lib.rs
        └── rules.star

Build Rules

In rules.star:

load("@prelude//rust:rust.bzl", "rust_library", "rust_binary")

rust_library(
    name = "mycrate",
    srcs = glob(["src/**/*.rs"]),
    deps = ["rustdeps//serde:serde"],
)

External Dependencies

Reference crates via the rustdeps cell:

deps = [
    "rustdeps//serde:serde",
    "rustdeps//tokio:tokio",
]

Features and Dependencies

Each vendored crate is built with the features and dependencies cargo resolves for it. tk sync records, for every crate, its package slice in rust-deps.toml: the features cargo tree reports for cargo test --workspace, and its normal dependencies resolved to exact versions, each with the platforms it applies on (one cargo tree run per platform the project builds for). Each crate's own rules.star is generated from its slice alone, so a change to one crate rebuilds only that crate and its dependents.

Declare what you need in Cargo.toml as you would for Cargo; there is nothing to repeat for Buck2, and no way to override Cargo's result: ask for a feature in the member's Cargo.toml.

  • tk sync runs cargo with --locked, so Cargo.lock must match Cargo.toml. After editing a manifest by hand, run a cargo command (or tw cargo …) that updates the lock. The first run downloads the crates into ~/.cargo/registry.
  • Every workspace member's Cargo.toml is a source of rust-deps.toml, so a features-only edit in a member regenerates it.
  • A crate no configured platform builds (Windows-only, wasm, a build dependency) has an empty slice: its rules.star has no features or dependencies, and nothing configures it.

The rustdeps cell is a real directory, .turnkey/rustdeps, kept in line with the Nix-built cell index by tk materialize; see Managing Dependencies.

Auto-Sync

The cargo command is wrapped to auto-sync:

cargo add serde  # Triggers sync

Python Support

Turnkey provides Python support with Buck2 integration.

Python source in this repo is laid out as a uv workspace, with each package owning its own pyproject.toml and contributing to a shared turnkey.* PEP 420 namespace. This page covers the Buck2 build rules; read the workspace workflow guide first for the overall layout and the uv/Buck2 dual-track model.

Setup

Add to toolchain.toml:

[toolchains]
python = {}
uv = {}
pydeps-gen = {}

Enable Python dependencies in flake.nix:

turnkey.toolchains.buck2.python = {
  enable = true;
  depsFile = ./python-deps.toml;
};

Project Structure

my-project/
├── pyproject.toml
├── uv.lock
├── python-deps.toml      # Generated from uv.lock
└── python/
    └── mypackage/
        ├── __init__.py
        ├── main.py
        └── rules.star

Build Rules

In rules.star:

load("@prelude//python:python.bzl", "python_library", "python_binary", "python_test")

python_library(
    name = "mypackage",
    srcs = glob(["**/*.py"]),
    deps = ["pydeps//requests:requests"],
)

python_binary(
    name = "main",
    main = "main.py",
    deps = [":mypackage"],
)

python_test(
    name = "test",
    srcs = ["test_main.py"],
    deps = [":mypackage"],
)

External Dependencies

Reference packages via the pydeps cell:

deps = [
    "pydeps//requests:requests",
    "pydeps//click:click",
]

python-deps.toml

pydeps-gen writes it from pylock.toml (what to fetch) and uv.lock (the dependency graph). Schema 3 records each distribution's pure wheel, with markers and extras:

schema_version = 3

[deps.requests]
version = "2.32.3"
# The Nix hash of the unpacked wheel
hash = "sha256-..."
# The locked py3-none-any wheel
url = "https://files.pythonhosted.org/.../requests-2.32.3-py3-none-any.whl"
# The lock's marker for installing the package at all, when it has one
marker = "python_version >= '3.8'"
# Its dependencies: each package's key, with its marker and the extras it
# asks for, when it has them
dependencies = [
    { name = "urllib3" },
    { name = "colorama", marker = "sys_platform == 'win32'" },
]
# The extras some package or workspace member asks it for
requested_extras = ["socks"]

# Its extras, and the dependencies each adds
[deps.requests.extras]
"socks" = [{ name = "pysocks" }]

The pydeps cell uses them: a package's target depends on its dependencies and on those of its requested extras, each where its marker holds. Markers are evaluated on every platform in buck2.platforms, for the Python toolchain's version: a dependency some platforms get is a select() on the platform, and one none gets is left out.

A file from before schema 3 records sdists, and fails evaluation asking for tk sync, which regenerates it.

Distributions are their locked wheels

The pydeps cell vendors each distribution as the pure (py3-none-any) wheel uv locked for it, unpacked: the layout an installer puts in site-packages (ADR 0013). So import requests works although requests keeps its code under src/ in its sdist, and an sdist's setup.py, tests/ and docs are not in the target.

  • The wheel's <name>.data/purelib and platlib are merged into its root, as an installer would; the rest of <name>.data/ (scripts, headers, data) is dropped. Fixups, then user patches, apply to that tree.
  • The distribution's python_library holds everything the wheel installs: its .py files as srcs, and every other file as resources. That includes data files (such as certifi's cacert.pem) and the *.dist-info directory, so importlib.metadata.version("requests") and entry points work.
  • A distribution with no pure wheel fails pydeps-gen, which names it: one with only platform-specific (compiled) wheels, and one with only an sdist. turnkey doesn't build sdists into wheels, and doesn't vendor platform wheels yet (#248).

src/examples/python-requests uses requests and certifi this way.

Markers and Extras

Rules sync reads a workspace member's pyproject.toml as well as its imports:

  • A dependency in [project] dependencies with a platform marker (sys_platform, platform_system, platform_machine, os_name) is written as a select() over the platforms, like any platform-conditional dep.
  • Other markers (python_version, implementation_name, ...) are fixed by the Python toolchain: they are evaluated once, for the python3 of the shell, and a dependency whose marker doesn't hold is left out.
  • Extras are a variant: a target declares the extras it's built with on turnkey's prelude attribute extras (possibly a select()), and sync adds the dependencies they enable, whether its sources import them or not. A dependency declared only as an extra's is kept only on a target built with that extra.
python_library(
    name = "app-full",
    extras = ["socks"],
    deps = [...],  # sync adds the socks extra's dependencies
)

Auto-Sync

The uv command is wrapped to auto-sync:

uv add requests  # Triggers sync

TypeScript Support

Turnkey provides TypeScript support via custom Buck2 rules.

Setup

Add to toolchain.toml:

[toolchains]
nodejs = {}
typescript = {}

Project Structure

my-project/
└── ts/
    └── myapp/
        ├── src/
        │   └── index.ts
        ├── tsconfig.json     # Optional
        └── rules.star

Build Rules

In rules.star:

load("@prelude//typescript:typescript.bzl", "typescript_binary", "typescript_library")

typescript_library(
    name = "lib",
    srcs = glob(["src/**/*.ts"]),
)

typescript_binary(
    name = "myapp",
    main = "src/index.ts",
    srcs = glob(["src/**/*.ts"]),
    deps = [":lib"],
)

Running TypeScript

tk run //ts/myapp:myapp

A typescript_test takes the same attributes as a typescript_binary. It compiles the code and runs it with node: the test passes when it exits 0.

typescript_test(
    name = "myapp_test",
    main = "src/index.test.ts",
    srcs = glob(["src/**/*.ts"]),
)
tk test //ts/myapp:myapp_test

Configuration

The TypeScript toolchain uses sensible defaults. For custom configuration, provide a tsconfig.json:

typescript_binary(
    name = "myapp",
    main = "src/index.ts",
    srcs = glob(["src/**/*.ts"]),
    tsconfig = "tsconfig.json",
)

npm Dependencies

npm packages are declared in the root package.json and locked in pnpm-lock.yaml. tk sync runs jsdeps-gen, which writes js-deps.toml, and the jsdeps cell is built from it (ADR 0012). Rules sync writes a target's npm_deps from the packages its sources import.

Labels

A target lists the packages it imports in its npm_deps, by their npm name, verbatim:

typescript_binary(
    name = "app",
    main = "main.ts",
    srcs = ["main.ts"],
    npm_deps = [
        "jsdeps//:lodash",
        "jsdeps//:@types/lodash",
    ],
)

Only the root package.json's dependencies (js-deps.toml's [direct]) have such a label, at the version the root resolves them to. Whatever they depend on is in the cell too, but code can't import it, just as pnpm keeps it out of the root node_modules: declare a package in package.json to import it. @types/... packages are usually devDependencies: set buck2.javascript.includeDevDependencies = true so they are direct too.

Labels used to name a scoped package with the @ dropped and / as _ (jsdeps//:types_lodash). Rules sync replaces them with the npm names (Upgrading).

The node_modules each target sees

Each locked name@version is fetched once, as its own store path. Each instance of it, one per pnpm snapshot, is a target in the cell: a package that pnpm installs once per peer resolution (react-dom with React 17 and with React 18) has one instance per resolution, and two versions of one name are two instances.

An instance's output is a real node_modules/<name>/ directory, copied from the package's files, with each of its dependencies a relative symlink beside it, into that dependency's instance. A target's node_modules links each of its npm_deps to its instance's package directory. node and tsc find a package's own dependencies by walking node_modules up from its real path, as in pnpm's node_modules/.pnpm, so each package sees exactly the dependencies its lock entry resolves, with no --preserve-symlinks and no NODE_PATH. A typescript_binary's output has its own node_modules link beside the compiled code, so node dist/main.js resolves the same way, for CommonJS (.cts, compiled to .cjs) and ES modules (.mts, to .mjs) alike.

Instances that depend on each other in a cycle are laid out together, as one target with a forwarding target per instance.

A bump of one package rebuilds its store path and re-runs only the instances that depend on it, and their consumers: .turnkey/jsdeps is a real directory that tk materialize keeps in line with the cell index, so plain buck2 reads the new version without a daemon restart.

Copying costs disk: every installed package is copied once per configuration under buck-out.

js-deps.toml

One [[package]] per locked name@version (its contents), one [[instance]] per pnpm snapshot (its dependencies, by the name it imports each as, resolved to instance keys), and [direct]:

[[package]]
name = "chokidar"
version = "3.6.0"
url = "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz"
integrity = "sha512-..."

[[package]]
name = "fsevents"
version = "2.3.3"
url = "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz"
integrity = "sha512-..."
os = ["darwin"]

[[instance]]
key = "chokidar@3.6.0"
name = "chokidar"
version = "3.6.0"

[instance.dependencies]
anymatch = "anymatch@3.1.3"
braces = "braces@3.0.3"

[instance.optional_dependencies]
fsevents = "fsevents@2.3.3"

[direct]
chokidar = "chokidar@3.6.0"
  • An instance's key is pnpm's snapshot key verbatim, peer groups included (react-dom@18.2.0(react@18.2.0)). Its target in the cell is named as pnpm names its directory in node_modules/.pnpm (react-dom@18.2.0_react@18.2.0).
  • os, cpu and libc are the package's own restrictions, as npm names them (darwin, x64, glibc, or !win32 to exclude one). A field that's absent allows anything.
  • link: and file: dependencies fail jsdeps-gen, naming the package.

Packages from a registry Nix can't reach

A package locked from a registry that Nix can't fetch from, such as a local one, can be kept in the project as its tarball, keyed by the URL pnpm-lock.yaml records for it. The cell reads the file instead of fetching, and still checks it against the lock's integrity:

buck2.javascript.tarballs = {
  "http://localhost:4873/-/acme-utils-1.0.0.tgz" = ./registry/acme-utils-1.0.0.tgz;
};

Platform-specific packages

An instance whose optional dependencies install on some platforms only, such as esbuild's per-platform binaries or chokidar's fsevents, is resolved in the cell, for every platform in buck2.platforms: the optional dependencies the platform gets (their os and cpu allow it, and on Linux their libc allows glibc) are linked beside it, as a select() keyed like rules sync's, with no DEFAULT. A consumer lists the package unconditionally. src/examples/typescript-platform-deps is an example.

Patching a package

User patches go under .turnkey/patches/jsdeps/vendor/<name>@<version>/, or vendor/<name>/ for the version a direct dependency resolves to, and apply to that package's files with --fuzz=0 (Fixups).

Solidity Support

Turnkey provides Solidity smart contract support with Buck2 integration, including compilation, testing with Foundry, and dependency management.

Setup

Add to toolchain.toml:

[toolchains]
solidity = {}
foundry = {}

Enable Solidity dependencies in flake.nix (if using external libraries):

turnkey.toolchains.buck2.solidity = {
  enable = true;
  depsFile = ./solidity-deps.toml;
};

Project Structure

my-project/
├── foundry.toml              # The only Foundry configuration, for the whole repository
├── solidity-deps.toml        # Generated dependency manifest
├── remappings.txt            # Generated from solidity-deps.toml
└── src/
    └── contracts/
        ├── rules.star
        ├── src/
        │   └── MyToken.sol
        └── test/
            └── MyToken.t.sol

The repository has at most one foundry.toml, at its root. A package under src/ has none of its own: a nested foundry.toml would become forge's root for its subtree and break native forge there. With the tk.foundryConfigCheck option on, a pre-commit hook rejects one (see Native forge).

Build Rules

forge drives both solidity_library and solidity_test, and reads the same configuration native forge reads. The root foundry.toml holds every compiler and test setting (optimizer, optimizer_runs, evm_version, via_ir, [fuzz] runs, ...), and the root remappings.txt every remapping (overrides go in foundry.toml, see Remappings). The rules take neither settings nor remappings as attributes. So tests run against the bytecode that ships, and native forge build produces the same bytecode as Buck2.

Each Solidity action runs forge in a scratch project that holds only its declared inputs, each at its place in the repository:

  • the root foundry.toml and remappings.txt;
  • the target's sources, and those of every solidity_library it depends on;
  • the soldeps cell's bundle (every vendor package), at the cell link's path, .turnkey/soldeps/, where the root remappings.txt points.

forge runs with the toolchain's solc (--use), --offline, and without the caller's FOUNDRY_*/DAPP_* variables or ~/.foundry configuration.

The solidity_library and solidity_test macros add these inputs themselves, from the [solidity] section turnkey writes into the generated .buckconfig: foundry.toml always, the bundle and remappings.txt when the repository has a soldeps cell. foundry.toml must be at the repository root (turnkey.toolchains.buck2.solidity.foundryTomlFile's default); the shell fails to evaluate otherwise.

Breaking change: a repository using the Solidity rules must export both files from a rules.star at its root:

# rules.star, at the repository root
export_file(name = "foundry.toml", visibility = ["PUBLIC"])
export_file(name = "remappings.txt", visibility = ["PUBLIC"])

Without them, a Solidity target fails to build with an unknown root//:foundry.toml target. The rules also no longer accept optimizer, optimizer_runs, fuzz_runs, solc_version or remappings: set them in the root foundry.toml.

Changing a setting in foundry.toml, or a dependency, rebuilds and retests every Solidity target.

solidity_library

Compile Solidity source files with forge build <srcs>:

load("@prelude//solidity:solidity.bzl", "solidity_library")

solidity_library(
    name = "my_token",
    srcs = ["src/MyToken.sol"],
    deps = ["soldeps//:openzeppelin_contracts"],
)

Its output is forge's artifacts directory (out/): one <File>.sol/<Contract>.json per contract compiled, imports included.

solidity_contract

Extract a specific contract from a compiled library:

load("@prelude//solidity:solidity.bzl", "solidity_contract")

solidity_contract(
    name = "my_token_artifact",
    contract = "MyToken",  # Contract name in source
    lib = ":my_token",
)

It takes the artifact of the contract of that name defined in one of the library's own sources (not in an import); if two of them define it, split the library. It produces what solc's own outputs would be:

  • {contract}.abi - Contract ABI (JSON)
  • {contract}.bin - Deployment bytecode (hex)
  • {contract}.bin-runtime - Runtime bytecode (hex)
  • {contract}.metadata.json - Compiler metadata

solidity_test

Run tests with Foundry's forge test:

load("@prelude//solidity:solidity.bzl", "solidity_test")

solidity_test(
    name = "my_token_test",
    srcs = ["test/MyToken.t.sol"],
    deps = [":my_token"],
)

Fuzz runs come from foundry.toml's [fuzz] runs. The fuzz seed is fixed per target, unless the target opts out of test result caching (see Test result caching).

External Dependencies

tk sync collects Solidity dependencies from two places, the root foundry.toml and the root package.json, into the generated solidity-deps.toml, and generates the root remappings.txt from that. Both files are committed. Only direct dependencies count: tk sync does not follow a dependency's own [dependencies], submodules or remappings.txt, so anything a dependency needs is declared at the root too.

Git dependencies

Declare git dependencies in the root foundry.toml's [dependencies], in Soldeer's table form:

[dependencies]
forge-std = { version = "1.8.0", git = "https://github.com/foundry-rs/forge-std", tag = "v1.8.0" }
solady = { version = "0.1.26", git = "https://github.com/vectorized/solady", rev = "<40-hex commit>" }
  • version is required. It is the package's version in solidity-deps.toml, and what fixup sets match on.
  • git is the repository URL.
  • The pin is exactly one of tag, branch or rev. A rev must be a full 40-character commit hash: tk sync resolves pins with git ls-remote, which lists refs but cannot expand an abbreviated commit without a clone. Pin a name with tag or branch instead.

Turnkey never runs Soldeer; soldeps-gen reads the table itself. A string value, such as a Soldeer registry version (forge-std = "1.9.7") or the older "<url>@<ref>" form, is rejected with an error showing the table form. So is Soldeer's url = "…" form, since URL dependencies are not supported, and any other unknown key, such as a misspelt pin.

npm packages

Solidity packages published to npm, such as @openzeppelin/contracts, are ordinary dependencies in the root package.json, at the versions and integrity hashes pnpm-lock.yaml pins. A package.json dependency counts as a Solidity dependency when its tarball contains .sol files; there is no list of known packages. To tell, tk sync downloads the tarball, checks it against the lock's integrity (and fails on a mismatch), and looks for .sol files, within generous size limits. solidity-deps.toml remembers the verdict for each lock integrity, or for each name and version when the lock records no integrity: a Solidity package is recorded as a [[package]], any other dependency as a [[not_solidity]] entry. A recorded verdict is always reused, so a sync only downloads the tarballs of packages that are new or that changed. When a download fails, or --no-prefetch rules it out, the sync fails instead of guessing.

Using a dependency

Reference a package through the soldeps cell:

solidity_library(
    name = "my_token",
    srcs = ["MyToken.sol"],
    deps = ["soldeps//:openzeppelin_contracts"],
)

and import it under its own name:

import "@openzeppelin/contracts/token/ERC20/ERC20.sol";
import "forge-std/Test.sol";

Syncing

tk sync regenerates solidity-deps.toml whenever foundry.toml, package.json or pnpm-lock.yaml changes (the paths are the turnkey.toolchains.buck2.solidity options foundryTomlFile, packageJsonFile and pnpmLockFile), then the root remappings.txt, next to foundry.toml, from solidity-deps.toml. It runs soldeps-gen, which prefetches: it pins each git dependency to the commit its tag, branch or rev resolves to. For a GitHub repository it also records that commit's source archive and its Nix hash, so the soldeps cell fetches it as a fixed-output derivation. An npm package that pnpm-lock.yaml gives no integrity for gets the hash of its tarball. Hashes go through turnkey's prefetch cache, so a regeneration only fetches what changed.

Remappings

Each package is imported under its own name, <name>/, with no aliases: forge-std/... resolves inside forge-std. An npm package maps to its root. A git dependency maps to the directory its own foundry.toml names as [profile.default] src, to forge's default src/ when its foundry.toml sets none, and to the repository root when it has no foundry.toml. soldeps-gen reads that file at the pinned commit while prefetching, and records the result as the package's remapping in solidity-deps.toml. Later syncs reuse the recorded target while the package's pin is unchanged, so a sync with --no-prefetch works for them; a new or re-pinned git dependency needs a prefetching sync.

soldeps-gen reads a dependency's foundry.toml only from GitHub. For a git dependency hosted elsewhere, the sync fails rather than guess; give it an override (below) naming where its sources live.

To point a package elsewhere, add a remapping to the root foundry.toml:

[profile.default]
remappings = ["solady/=.turnkey/soldeps/vendor/solady/src/"]

Its prefix must be <name>/ of a declared package (remappings cannot alias one package under another name), and its target must lie inside .turnkey/soldeps/vendor/<name>/. tk sync rejects anything else.

The root remappings.txt sits next to foundry.toml and holds every package's remapping, overrides included, with targets under the soldeps cell link. Forge resolves targets from foundry.toml's directory, so with the file at the project root they read:

@openzeppelin/contracts/=.turnkey/soldeps/vendor/@openzeppelin/contracts/
forge-std/=.turnkey/soldeps/vendor/forge-std/src/

With a foundry.toml in a subdirectory, the targets (and the overrides you write) start with one ../ per level instead. Forge prefers remappings.txt to the remappings key, and the two agree because the file already contains the overrides. Don't edit the file; change foundry.toml and run tk sync.

Native forge

forge build and forge test work in the dev shell alongside tk build and tk test, the same way running cargo or go natively does. They run from any directory, since forge finds the root foundry.toml, and cover every package under src/:

[profile.default]
src = "src"
test = "src"
libs = [".turnkey/soldeps/vendor"]
out = "out"
auto_detect_remappings = false
optimizer = true
optimizer_runs = 200

[fuzz]
runs = 256

src and test are both src, so Solidity code added anywhere under src/ is covered without editing the file. There is no per-package scoping; narrow a run with --match-path instead:

forge build
forge test --match-path 'src/contracts/**'

Git-ignore forge's /out/ and /cache/.

Imports resolve through the root remappings.txt that tk sync generates (see Remappings), into the soldeps cell's vendor/ directory (libs), where each package's files are linked beside its alias package (see The Go, Rust and Solidity Cells Are Real Directories). Automatic remapping detection is off, so forge does not guess remappings of its own.

The compiler comes from the dev shell. foundry.toml sets neither solc nor solc_version. When Solidity is enabled, the dev shell exports FOUNDRY_SOLC, the solc the Buck2 toolchain uses, together with FOUNDRY_OFFLINE=true. Native runs therefore share their compiler with the Buck2 rules and never download one; the other settings are shared through foundry.toml itself (see Build Rules). The toolchain declared in toolchain.toml stays the one place the compiler version is set; a solc_version in foundry.toml could only go stale on a toolchain bump, so the pre-commit hook rejects solc and solc_version keys.

Compiler Version

The compiler version comes from the toolchain declared in toolchain.toml (solidity-toolchain or solc, not both): Buck2's Solidity rules and native forge (through FOUNDRY_SOLC) both use its solc. To change the version, change the toolchain.

There is one compiler per repository: the rules take no per-target version.

Building and Testing

# Build contracts
tk build //src/contracts:my_token

# Run tests
tk test //test:my_token_test

# Build all Solidity targets
tk build //... --target-platforms //platforms:solidity

Forge Integration

The solidity_test rule wraps Foundry's forge test, supporting:

  • Unit tests
  • Fuzz testing
  • Fork testing (with fork_url attribute)
  • Gas reports
solidity_test(
    name = "integration_test",
    srcs = ["Integration.t.sol"],
    deps = [":my_token"],
    fork_url = "https://eth-mainnet.g.alchemy.com/v2/...",  # Optional
)

Jsonnet Support

Turnkey provides Jsonnet support for generating JSON configuration files with Buck2 integration. Jsonnet is a data templating language that extends JSON with variables, functions, and imports.

Setup

Add to toolchain.toml:

[toolchains]
jsonnet = {}

Turnkey uses jrsonnet, a fast Rust implementation of Jsonnet.

Project Structure

my-project/
├── config/
│   ├── base.libsonnet       # Shared configuration
│   ├── dev.jsonnet          # Development config
│   ├── prod.jsonnet         # Production config
│   └── rules.star

Build Rules

jsonnet_library

Compile Jsonnet files to JSON:

load("@prelude//jsonnet:jsonnet.bzl", "jsonnet_library")

jsonnet_library(
    name = "config-dev",
    srcs = ["dev.jsonnet"],
    deps = [":base"],  # Dependencies on other jsonnet_library targets
    ext_strs = {
        "env": "development",
        "region": "us-west-2",
    },
)

Attributes

AttributeDescription
srcsJsonnet source files (first file is entry point)
depsDependencies on other jsonnet_library targets
outOutput filename (defaults to <src>.json)
ext_strsExternal string variables (--ext-str key=value)
ext_codesExternal code variables (--ext-code key=value)
tla_strsTop-level argument strings (--tla-str key=value)
tla_codesTop-level argument code (--tla-code key=value)

Example

base.libsonnet

{
  // Shared configuration
  appName: 'my-app',
  version: '1.0.0',

  // Environment-specific overrides
  envConfig(env):: {
    development: {
      logLevel: 'debug',
      replicas: 1,
    },
    production: {
      logLevel: 'warn',
      replicas: 3,
    },
  }[env],
}

dev.jsonnet

local base = import 'base.libsonnet';
local env = std.extVar('env');

base {
  environment: env,
  config: base.envConfig(env),
}

rules.star

load("@prelude//jsonnet:jsonnet.bzl", "jsonnet_library")

# Shared library
jsonnet_library(
    name = "base",
    srcs = ["base.libsonnet"],
)

# Development config
jsonnet_library(
    name = "config-dev",
    srcs = ["dev.jsonnet"],
    deps = [":base"],
    ext_strs = {"env": "development"},
)

# Production config
jsonnet_library(
    name = "config-prod",
    srcs = ["dev.jsonnet"],  # Same template, different vars
    deps = [":base"],
    ext_strs = {"env": "production"},
    out = "config-prod.json",
)

Building

# Build a specific config
tk build //config:config-dev

# View the output
tk build //config:config-dev --show-output
cat $(tk build //config:config-dev --show-output 2>&1 | grep -o 'buck-out/[^ ]*')

# Build all configs
tk build //config:...

External Variables

ext_strs (External Strings)

Pass string values from the build system:

jsonnet_library(
    name = "config",
    srcs = ["config.jsonnet"],
    ext_strs = {
        "env": "production",
        "version": "1.2.3",
    },
)

Access in Jsonnet:

{
  environment: std.extVar('env'),
  version: std.extVar('version'),
}

ext_codes (External Code)

Pass Jsonnet expressions:

jsonnet_library(
    name = "config",
    srcs = ["config.jsonnet"],
    ext_codes = {
        "replicas": "3",
        "features": "['auth', 'api']",
    },
)

Top-Level Arguments

For parameterized configs using functions:

// config.jsonnet
function(env, replicas=1) {
  environment: env,
  replicas: replicas,
}
jsonnet_library(
    name = "config",
    srcs = ["config.jsonnet"],
    tla_strs = {"env": "production"},
    tla_codes = {"replicas": "5"},
)

Use Cases

  • Kubernetes manifests - Generate YAML/JSON configs with environment-specific values
  • Application configuration - Type-safe config generation with inheritance
  • Infrastructure as Code - Generate Terraform JSON, CloudFormation, etc.
  • CI/CD pipelines - Generate pipeline configs from templates

CLI Reference

This reference covers all Turnkey CLI commands.

tk - Buck2 Wrapper

tk is the primary Turnkey CLI. It wraps Buck2 with automatic dependency synchronization.

Overview

When using Buck2 with Nix-managed dependencies, certain files must be regenerated when source files change. tk solves this by automatically running sync operations before buck2 commands that read the build graph.

Quick Start

# Use tk just like buck2 - it syncs automatically
tk build //some:target     # syncs first, then builds
tk test //some:target      # syncs first, then tests
tk run //some:target       # syncs first, then runs

# Explicit sync operations
tk sync                    # manually sync all stale files
tk check                   # check if files are stale (for CI)

# Skip sync when needed
tk --no-sync build //...   # skip sync, run buck2 directly

Command Reference

tk build/run/test/... (Buck2 passthrough)

Most tk commands are passed through to Buck2. Commands that read the build graph automatically sync first:

Sync-first commands (sync before running):

  • build - Build targets
  • run - Run a target
  • test - Run tests
  • query - Query the build graph
  • cquery - Configured query
  • uquery - Unconfigured query
  • targets - List targets
  • audit - Audit the build
  • bxl - Run BXL scripts

Pass-through commands (no sync):

  • clean - Clean build artifacts
  • kill - Kill Buck2 daemon
  • killall - Kill all Buck2 processes
  • status - Show daemon status
  • log - View build logs
  • rage - Generate debug report
  • help - Show help
  • docs - Open documentation
  • init - Initialize a project

Unknown commands default to syncing first (safe default).

tk sync

Explicitly synchronize all stale files, or only those of the named deps rules (go, rust, pylock, python, javascript, solidity; the rules are in .turnkey/sync.toml). Rules always run in sync.toml order, so pylock runs before python, which reads what it writes.

tk sync              # sync stale files
tk sync go           # sync only go-deps.toml
tk sync --verbose    # show what's being synced
tk sync --dry-run    # show what would be synced without doing it

Exit codes:

  • 0 - Success (files synced or nothing to sync)
  • 1 - Sync failed

tk check

Check if any files are stale without regenerating them. Useful for CI validation.

tk check             # check staleness
tk check rust        # check only rust-deps.toml
tk check --verbose   # show detailed status

Exit codes:

  • 0 - All files up-to-date
  • 1 - Files are stale (run tk sync to fix)

Example CI usage:

- name: Check files in sync
  run: tk check

tk completion

Generate shell completion scripts.

tk completion bash    # output bash completion script
tk completion zsh     # output zsh completion script
tk completion fish    # output fish completion script

Enable completions:

# Bash (add to ~/.bashrc)
eval "$(tk completion bash)"

# Zsh (add to ~/.zshrc)
eval "$(tk completion zsh)"

# Fish (run once)
tk completion fish > ~/.config/fish/completions/tk.fish

tk materialize

Bring deps cells in line with their cell indexes. The shell runs it on every load (direnv's use_turnkey and enterShell), so you rarely need to.

tk materialize /nix/store/…-rustdeps-index.json

It adds missing store links, rewrites alias packages whose target changed, removes what the index no longer names, roots the index under .turnkey/gcroots/, and records the deps file's hash. It fails, and changes nothing, if a store link points anywhere but its own name. A run with nothing to do touches no file.

tk rules

Manage rules.star files that define Buck2 build targets from source files. This command automatically detects imports from source files and updates the deps list in rules.star.

tk rules check              # Check every rules.star file against its sources
tk rules sync               # Update rules.star files with detected dependencies
tk rules help               # Show help

Options:

FlagDescription
--all, -async: process all files (skip staleness detection)
--force, -fSame as --all
--verbose, -vShow detailed output including skipped files
--quiet, -qSuppress output
--dry-run, -nShow what would be changed without writing

Staleness Detection:

check always checks every rules.star, so it also catches a stale file that is already committed. sync by default only processes directories with uncommitted changes whose source files are newer than rules.star. Use --all or --force to sync all files.

Examples:

tk rules check                    # Check all rules.star files
tk rules check src/cmd/tk         # Check one directory
tk rules sync                     # Update stale rules.star files
tk rules sync --all               # Force update all files
tk rules sync src/cmd/tk          # Sync specific directory
tk rules sync --dry-run           # Preview changes without writing

Preserving Manual Dependencies:

If you have manual dependencies that shouldn't be auto-detected, use preservation markers in your rules.star:

# turnkey:preserve-start
    "//some/manual:dep",
# turnkey:preserve-end

Dependencies within these markers are preserved during sync.

How tk runs it: tk rules, and the rules sync tk runs before a Buck2 command, call the rules-syncer library (src/rust/rules-syncer) linked into tk, and print what it reports as above.

tk Flags

Flags must come before the subcommand:

FlagDescription
--no-syncSkip sync and the cell-freshness check, run Buck2 directly
--no-localSkip local target overrides from .turnkey/local.toml
--verbose, -vShow what tk is doing
--dry-run, -nShow what would be synced without doing it
--quiet, -qSuppress non-error output
--help, -hShow help

Examples:

tk --no-sync build //...     # skip sync
tk --no-local run //target   # skip local overrides
tk --verbose sync            # verbose sync
tk -v -n sync                # dry-run with verbose output

Configuration

Sync Configuration File

tk reads staleness rules from .turnkey/sync.toml. This file is automatically generated from your Nix configuration.

When you configure dependency files in your flake.nix:

  • goDepsFile generates a Go deps rule
  • rustDepsFile generates a Rust deps rule
  • pythonDepsFile generates a Python deps rule

Example generated sync.toml:

[[deps]]
name = "go"
sources = ["go.mod", "go.sum"]
target = "go-deps.toml"
generator = ["godeps-gen", "--go-mod", "go.mod", "--go-sum", "go.sum"]

[[deps]]
name = "rust"
sources = ["Cargo.toml", "Cargo.lock"]
target_sources = "manifests"
target = "rust-deps.toml"
generator = ["rustdeps-gen", "--cargo-lock", "Cargo.lock", "--platform", "linux-x86_64=x86_64-unknown-linux-gnu", "--platform", "macos-arm64=aarch64-apple-darwin"]

Each [[deps]] entry defines:

  • name - Human-readable name for this rule
  • sources - Files that trigger regeneration when modified
  • target - The generated file
  • generator - Command to regenerate the target
  • target_sources (optional) - A top-level key of the target whose array lists more sources, relative to the project root. They are files only the generator can find, such as a Cargo workspace's member manifests. A target without that key is stale.

Local Target Overrides

tk supports per-developer local overrides via .turnkey/local.toml. This file is not committed to git, allowing each developer to customize target arguments for their local environment.

Use cases:

  • Different network addresses for local development
  • Debug flags for specific targets
  • Custom ports or configuration

Example .turnkey/local.toml:

# Override args for tk run
[run."//docs/user-manual"]
args = ["-n", "192.168.1.100"]

# Override args for tk build
[build."//src/cmd/server:server"]
args = ["--config=debug"]

# Pattern matching with "..."
[test."//src/..."]
args = ["--verbose", "--timeout=60s"]

How it works:

When you run a command that matches a configured target:

tk run //docs/user-manual
# Becomes: buck2 run //docs/user-manual -- -n 192.168.1.100

The args are injected after --, which passes them to the target binary.

Pattern matching:

Patterns ending with ... match any target with that prefix:

  • //src/... matches //src:foo, //src/pkg:bar, //src/cmd/tool:main
  • //... matches any target

Disable for a single command:

tk --no-local run //docs/user-manual   # skips local.toml

Verbose output:

tk --verbose run //docs/user-manual
# Output: tk: applying local override for run //docs/user-manual: [-n 192.168.1.100]

Shell Integration

buck2 alias to tk:

# In devenv shell, buck2 is aliased to tk
buck2 build //...   # actually runs: tk build //...

Disable with:

TURNKEY_NO_ALIAS=1 buck2 build //...   # uses raw buck2

tw - Native Tool Wrapper

tw wraps native language tools (go, cargo, uv) to keep dependency files in sync when using standard workflows.

The Problem

When you run go get github.com/foo/bar, Go updates go.mod and go.sum. But Buck2 needs go-deps.toml to know about the new dependency. Without auto-sync, you'd need to manually regenerate it.

The Solution

Turnkey transparently wraps go, cargo, and uv so that dependency sync happens automatically:

go get github.com/foo/bar  # Just works - go-deps.toml is auto-updated

How It Works

User runs: go get github.com/foo/bar
                    │
                    ▼
Shell wrapper (provides 'go' binary)
• Sets TURNKEY_REAL_GO to actual go binary path
• Calls: tw go get github.com/foo/bar
                    │
                    ▼
tw (turnkey wrapper)
1. Loads .turnkey/sync.toml configuration
2. Finds the [[wrappers]] rule for 'go' (none: runs go untouched)
3. Checks if 'get' is a mutating subcommand → yes
4. Captures SHA256 hashes of go.mod, go.sum
5. Runs the real 'go get' command
6. Compares hashes - detects changes
7. Runs the rule's post-commands (go mod tidy)
8. Runs the go deps rule (godeps-gen → go-deps.toml), then any other
   rule left stale

Supported Tools

turnkey writes one [[wrappers]] rule per language into .turnkey/sync.toml, from its language records (nix/buck2/languages.nix), for each enabled language that has deps rules:

ToolMutating CommandsWatch FilesDeps Rule
goget, modgo.mod, go.sumgo (go-deps.toml)
cargoadd, remove, updateCargo.toml, Cargo.lockrust (rust-deps.toml)
uvadd, remove, lock, syncpyproject.toml, uv.lockpylock (pylock.toml), then python (python-deps.toml)

The file names are the configured ones (buck2.go.modFile and so on). Without buck2.python.uvLockFile, uv watches only pyproject.toml and runs the python rule.

Escape Hatches

Bypass for a Single Command

TURNKEY_NO_WRAP=1 go get github.com/foo/bar

This runs the real go directly, skipping tw entirely.

Disable Sync for a Command

tw --no-sync go get github.com/foo/bar

This runs through tw but skips the sync step even if files change.

Verbose Output

tw -v go get github.com/foo/bar

Shows what tw is doing:

tw: capturing state of [go.mod go.sum]
tw: detected changes in [go.mod go.sum], running sync
Syncing go-deps.toml...
  Running: godeps-gen --go-mod go.mod --go-sum go.sum
  Regenerated go-deps.toml

Non-Mutating Commands

Commands not in mutating_subcommands pass through without any overhead:

go build ./...    # No snapshot, no sync check - just runs go build
go version        # Direct passthrough

Deps generators

tk sync runs a deps generator for each enabled language (the [[deps]] rules of .turnkey/sync.toml), so you rarely run one yourself. When you do, every generator takes the same options:

OptionDescription
-o, --output PATHOutput file (default: stdout)
--no-prefetchSkip prefetching the Nix hashes the deps cell fetches with (the file gets placeholder or missing hashes)
--no-cacheAlways fetch from the network, bypassing turnkey's prefetch cache

Prefetching is on by default. jsdeps-gen takes only --output: the pnpm lock's integrity hashes are the ones Nix fetches with.

godeps-gen

Generate go-deps.toml from go.mod and go.sum.

Usage

godeps-gen [OPTIONS]

Options

OptionDescription
--go-mod PATHPath to go.mod file (default: go.mod)
--go-sum PATHPath to go.sum file (default: go.sum)
--indirectInclude indirect dependencies (default: true)

Prefetching hashes the modules' proxy.golang.org zips, the source the godeps cell fetches, in one nix-prefetch-cached --batch call: turnkey's prefetch cache answers the modules it has seen, and the rest are fetched in parallel.

Examples

# Generate with prefetched hashes
godeps-gen -o go-deps.toml

# Use custom paths
godeps-gen --go-mod src/go.mod --go-sum src/go.sum -o go-deps.toml

# Quick check without fetching (placeholder hashes)
godeps-gen --no-prefetch

rustdeps-gen

Generate rust-deps.toml from Cargo.lock.

Usage

rustdeps-gen [OPTIONS]

Options

OptionDescription
--cargo-lock PATHPath to Cargo.lock file (default: Cargo.lock)
--cargo-toml PATHPath to the workspace's root Cargo.toml (default: next to Cargo.lock)

Examples

# Generate from default Cargo.lock
rustdeps-gen -o rust-deps.toml

# Use custom path
rustdeps-gen --cargo-lock rust/Cargo.lock -o rust-deps.toml

pydeps-gen

Generate python-deps.toml from Python dependency files. Each distribution is recorded as its pure (py3-none-any) wheel, hashed unpacked; a distribution with only platform-specific wheels, or only an sdist, fails with its name (see Python).

Usage

pydeps-gen [OPTIONS]

Options

OptionDescription
--lock PATHPath to pylock.toml (PEP 751 lock file) - RECOMMENDED
--pyproject PATHPath to pyproject.toml
--requirements PATHPath to requirements.txt
--uv-lock PATHuv.lock, whose dependency graph is recorded with the --lock packages
--include-devInclude dev dependencies

Input Formats

FormatFlagReproducibilityNotes
pylock.toml (PEP 751)--lockBestExact versions and URLs
pyproject.toml--pyprojectVariesUses latest matching versions
requirements.txt--requirementsVariesPin versions with ==
# 1. Generate lock file from pyproject.toml
uv lock

# 2. Export to PEP 751 format
uv export --format pylock.toml -o pylock.toml

# 3. Generate python-deps.toml with Nix hashes
pydeps-gen --lock pylock.toml -o python-deps.toml

Examples

# From PEP 751 lock file (best for reproducibility)
pydeps-gen --lock pylock.toml -o python-deps.toml

# From pyproject.toml (resolves to latest matching versions)
pydeps-gen --pyproject pyproject.toml -o python-deps.toml

# From requirements.txt
pydeps-gen --requirements requirements.txt -o python-deps.toml

# Include dev dependencies
pydeps-gen --lock pylock.toml --include-dev -o python-deps.toml

Troubleshooting

"tk: .buckconfig not found"

tk looks for .buckconfig to find the project root. Make sure you're in a Buck2 project directory.

"tk: failed to load sync config"

The .turnkey/sync.toml file is missing or invalid. Ensure you're in a Turnkey project with proper configuration.

"tk: sync failed: generator command failed"

The generator command failed. Check that:

  • The generator command is correct
  • Required tools are in PATH
  • Source files exist

Sync is slow

If sync takes a long time:

  • Prefetched hashes are cached (prefetch-cache.json under $TURNKEY_CACHE_DIR, or else turnkey/ in the platform cache directory, e.g. ~/.cache/turnkey/ on Linux); check a generator isn't running with --no-cache
  • Check if generators are doing unnecessary work

Bypass tk

If you need to use raw Buck2:

# Option 1: --no-sync flag
tk --no-sync build //...

# Option 2: TURNKEY_NO_ALIAS environment variable
TURNKEY_NO_ALIAS=1 buck2 build //...

Troubleshooting

Common issues and solutions when using Turnkey.

Shell Issues

"attribute 'X' missing" when entering shell

Cause: A toolchain in toolchain.toml isn't in the registry.

Solution: Either remove the toolchain from toolchain.toml or add it to your registry in flake.nix.

Changes to toolchain.toml not taking effect

Cause: Nix flake caching.

Solution:

  1. Stage changes: git add toolchain.toml
  2. Re-enter shell: exit && nix develop

Build Issues

"toolchain not found" error

Cause: The language toolchain wasn't generated.

Solution: Ensure the toolchain is:

  1. Declared in toolchain.toml
  2. Has a mapping in nix/buck2/mappings.nix (for custom toolchains)

"missing BUCK file" or "missing rules.star"

Cause: Buck2 can't find build files.

Solution: Check that:

  1. .buckconfig has [buildfile] name = rules.star
  2. All cells have proper .buckconfig with buildfile settings

Stale dependency errors

Cause: Dependency cells out of sync with lock files.

Solution:

tk sync
tk build //...

Dependency Issues

godeps cell missing packages

Cause: go-deps.toml out of date.

Solution:

godeps-gen > go-deps.toml
git add go-deps.toml
# Re-enter shell

A vendored Rust crate's features differ from what you expect

Cause: Each crate gets the features cargo resolves for cargo test --workspace, per platform, recorded in rust-deps.toml by tk sync. Either rust-deps.toml is out of date, or a member doesn't ask for the feature.

Solution:

  1. Regenerate it: tk sync. A features-only edit to a member's Cargo.toml is enough to make the Rust rule stale.
  2. Compare with cargo itself: cargo tree --target <triple> -e normal,dev -i <crate> --format '{p} {f}'.
  3. Ask for the feature in the member's Cargo.toml that uses the crate. There is no override file: rust-features.toml is retired (see Upgrading).

FUSE Issues

FUSE not available on Linux

Cause: FUSE kernel module not loaded or /dev/fuse missing.

Solution:

# Load FUSE module
sudo modprobe fuse

# Verify
ls /dev/fuse

If persistent, add fuse to /etc/modules-load.d/.

"FUSE-T not installed" on macOS

Cause: FUSE-T package not installed.

Solution:

brew install macos-fuse-t/homebrew-cask/fuse-t

Mount point already in use

Cause: Previous daemon didn't unmount cleanly.

Solution:

# Force unmount
tk compose down --force

# Or manually
fusermount3 -uz /firefly/myproject  # Linux
umount -f /firefly/myproject         # macOS

"Permission denied" on mount

Cause: User not in fuse group or mount point permissions.

Solution:

# Add user to fuse group (Linux)
sudo usermod -aG fuse $USER
# Log out and back in

# Check mount point permissions
sudo mkdir -p /firefly/myproject
sudo chown $USER:$USER /firefly/myproject

Daemon won't start

Cause: Various issues with daemon lifecycle.

Solution:

# Check for existing processes
pgrep -f turnkey-composed

# Kill stale processes
pkill -9 -f turnkey-composed

# Remove stale socket
rm -f /run/turnkey-composed/*.sock

# Start with debug logging
TURNKEY_FUSE_DEBUG=1 tk compose up

Files appear stale or missing

Cause: Dependency cells updating or policy blocking access.

Solution:

# Check daemon status
tk compose status

# Force refresh
tk compose refresh

# If in "building" state, wait or use lenient policy
TURNKEY_ACCESS_POLICY=lenient tk build //...

"Resource temporarily unavailable" (EAGAIN)

Cause: CI policy returning errors during updates.

Solution:

  • Wait for the build to complete
  • Switch to development policy for interactive use
  • Add retry logic in CI scripts

Build hangs waiting for FUSE

Cause: Strict policy blocking during long Nix builds.

Solution:

# Check what's blocking
tk compose status --verbose

# Use lenient policy for quick iteration
TURNKEY_ACCESS_POLICY=lenient tk build //...

# Or increase timeout
TURNKEY_BLOCK_TIMEOUT=600 tk build //...

Edits not persisting after restart

Cause: Edits stored in overlay, need to generate patches.

Solution:

# Generate patches before stopping
tk compose patch

# Then stop
tk compose down

Container/Docker issues

Cause: FUSE requires privileged access in containers.

Solution:

# Run container with FUSE access
docker run --device /dev/fuse --cap-add SYS_ADMIN ...

# Or disable FUSE and use symlinks
TURNKEY_FUSE_BACKEND=symlink tk build //...

Getting Help

  • Check GitHub Issues
  • Enable verbose mode: TURNKEY_VERBOSE=1 nix develop
  • Check Buck2 logs: tk log show
  • FUSE debug logs: TURNKEY_FUSE_DEBUG=1 tk compose up

Upgrading

What changes when a project moves to a newer turnkey, and what to do about it.

Python distributions are their locked wheels

Each pydeps distribution is now its locked pure (py3-none-any) wheel, unpacked, instead of its sdist (ADR 0013). Its target holds what an installer would put in site-packages: no setup.py or sdist tests/, src-layout distributions such as requests import under their own name, and data files and *.dist-info are resources. Labels are unchanged.

Switching over

  • Run tk sync. python-deps.toml moves to schema_version = 3, whose url and hash are the wheel's. A schema 2 file fails evaluation and asks for tk sync.
  • A distribution with no pure wheel fails pydeps-gen, which names it: one with only platform-specific wheels, or only an sdist. Such a distribution can't be vendored until platform wheels are supported (#248).
  • User patches against the sdist layout must be regenerated. Paths change with the layout (vendor/requests/src/requests/... becomes vendor/requests/requests/...), so such a patch no longer applies and fails its distribution's build. Make the change again in the materialized cell and write the patch with tk compose patch.
  • Fixups apply to the installed tree. Nothing runs an sdist build, so a fixup that relied on one (or on the sdist's paths) needs rewriting against the wheel's layout.

The per-distribution Python cell

The Python dependency cell is now built one package per distribution, and laid out by tk materialize as a real directory, .turnkey/pydeps, instead of one symlink to a merged cell (ADR 0004, ADR 0010). Labels (pydeps//vendor/<name>:<name>) are unchanged. A distribution bump now rebuilds that distribution alone, plus those whose dependencies name it if theirs changed, and re-runs only its dependents' actions, with no daemon restart.

Switching over

  • The first shell load after upgrading replaces the .turnkey/pydeps symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing.
  • A lock holding several versions of one distribution fails. pydeps-gen now rejects a uv resolution that forks on a marker, since the cell holds one version per name: pin the dependency so the lock no longer forks (see Python Workspaces).
  • User patches move. They go under .turnkey/patches/pydeps/vendor/<name>/, one directory per distribution, and apply in that distribution's own derivation. A patch file directly under .turnkey/patches/pydeps/ fails evaluation with where to move it: move it into the directory of the distribution whose files it changes, keeping its content. A patch spanning two distributions is split into one per distribution.

Rolling back

rm -rf .turnkey/pydeps .turnkey/pydeps.lock .turnkey/gcroots/pydeps

The JavaScript cell's package graph

The JavaScript dependency cell now holds each locked package once, at vendor/<name>@<version>, and one target per pnpm snapshot, laid out as pnpm lays out node_modules/.pnpm (ADR 0012). A package's own dependencies resolve without the consumer declaring them, two versions of one name and peer resolutions coexist, and dependency cycles build. tk materialize lays the cell out as a real directory, .turnkey/jsdeps, so a package bump re-runs only what depends on it, with no daemon restart, and plain buck2 reads the new version.

Switching over

  • Labels are npm names. jsdeps//:types_lodash is now jsdeps//:@types/lodash. Rules sync rewrites the labels it manages; change those in a turnkey:preserve section, or in a target sync doesn't manage, by hand.
  • Only direct dependencies have labels: the root package.json's (js-deps.toml's [direct]). A target that listed a package only some dependency needs drops it. @types/... packages are usually devDependencies: set buck2.javascript.includeDevDependencies = true.
  • Regenerate js-deps.toml with tk sync: the cell needs its [[instance]] and [direct] tables, which need a pnpm 9 lockfile.
  • The first shell load replaces the .turnkey/jsdeps symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one.
  • User patches move under .turnkey/patches/jsdeps/vendor/<name>@<version>/. A patch file directly under .turnkey/patches/jsdeps/ fails evaluation. See Dependency Fixups.

Rolling back

rm -rf .turnkey/jsdeps .turnkey/jsdeps.lock .turnkey/gcroots/jsdeps

The per-package Solidity cell

The Solidity dependency cell is now built one package per Solidity package, and laid out by tk materialize as a real directory, .turnkey/soldeps, instead of one symlink to a merged cell (ADR 0004, ADR 0011). Labels (soldeps//:<package>, soldeps//:bundle), the root remappings.txt and native forge are unchanged. A package bump no longer restarts the buck2 daemon: it re-runs the Solidity actions, and nothing in another language. Plain buck2 reads the new version.

Switching over

  • The first shell load after upgrading replaces the .turnkey/soldeps symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing.
  • A name declared twice (in foundry.toml and package.json, or in both dependencies and devDependencies) must resolve to one package: tk sync writes it once, or fails naming each declaration when they differ. The cell holds one version per name.
  • User patches move. They go under .turnkey/patches/soldeps/vendor/<name>/, one directory per package, and apply in that package's own derivation. A patch file directly under .turnkey/patches/soldeps/ fails evaluation with where to move it. See Dependency Fixups.

Rolling back

rm -rf .turnkey/soldeps .turnkey/soldeps.lock .turnkey/gcroots/soldeps

The per-module Go cell

The Go dependency cell is now built one package per module, and laid out by tk materialize as a real directory, .turnkey/godeps, instead of one symlink to a merged cell (ADR 0004, ADR 0008). Labels (godeps//vendor/<import path>:<last component>) are unchanged. A module bump now rebuilds that module alone and recompiles only its dependents: 8 actions for a one-module bump in turnkey's own repo.

Switching over

  • The first shell load after upgrading replaces the .turnkey/godeps symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing.
  • User patches move. They go under .turnkey/patches/godeps/vendor/<module path>/, one directory per module, and apply in that module's own derivation. A patch file directly under .turnkey/patches/godeps/ fails evaluation with where to move it: move it into the directory of the module whose files it changes, keeping its content. A patch spanning two modules is split into one per module.

Rolling back

rm -rf .turnkey/godeps .turnkey/godeps.lock .turnkey/gcroots/godeps

The per-crate Rust cell

The Rust dependency cell is now built one package per crate, and laid out by tk materialize as a real directory, .turnkey/rustdeps, instead of one symlink to a merged cell (ADR 0004). Each crate's features and dependencies come from cargo itself (ADR 0005, ADR 0006). A dependency bump now recompiles only the changed crates' dependents and re-runs only their tests: about 90 actions for a one-crate bump in turnkey's own repo, against about 1,200 before.

Switching over

  • The first shell load after upgrading regenerates rust-deps.toml (schema 2, with each crate's package slice) and replaces the .turnkey/rustdeps symlink with the directory. tk restarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing.
  • tk sync now runs cargo (cargo tree and cargo metadata, with --locked): Cargo.lock must match your Cargo.toml files, and the first run downloads the crates into ~/.cargo/registry.
  • Every workspace member's Cargo.toml is a source of rust-deps.toml, so a features-only edit in a member regenerates it.

Rolling back

To go back to an older turnkey, remove the directory before reloading the shell, so that the older shell can create its symlink again:

rm -rf .turnkey/rustdeps .turnkey/rustdeps.lock .turnkey/gcroots/rustdeps

rust-features.toml is retired

buck2.rust.featuresFile no longer exists, and setting it fails evaluation with a message pointing here. Its overrides are dropped: each crate gets the features cargo resolves for your workspace.

  • To get a feature, ask for it in the Cargo.toml of the member that uses the crate, for example serde = { version = "1", features = ["derive"] }.
  • Then remove the option and delete rust-features.toml.

[[requested]] is gone from rust-deps.toml too; regenerating it removes it.

Features may differ slightly

The old resolver was turnkey's own; the new one is cargo's. In turnkey's own repo, four of 317 crates changed, all to what cargo build does:

  • a crate no longer gets dependencies that only apply on targets you don't build (jiff's portable-atomic);
  • features that only one platform enables are set on that platform only (mio's log, zerocopy's derive);
  • features no configured platform enables are gone (tokio's windows-sys).

A crate that no configured platform builds (Windows-only, wasm, a build dependency) gets no features or dependencies, since nothing configures it.

User patches

Patches that tk compose patch writes for the Rust cell now live in their crate's directory, .turnkey/patches/rustdeps/vendor/<crate>@<version>/, and apply in that crate's own package. A patch file left directly under .turnkey/patches/rustdeps/ fails evaluation: move it into its crate's directory, or regenerate it with tk compose patch. See Dependency Fixups.