Introduction
Turnkey is a toolchain management framework for Nix flakes that simplifies declaring and managing build tools in development environments.
What is Turnkey?
Turnkey bridges declarative TOML configuration with Nix package resolution, providing:
- Simple Configuration: Declare toolchains in
toolchain.toml - Reproducible Environments: Nix ensures consistent tool versions across machines
- Incremental Builds: Fast, cached builds that only rebuild what changed
- Language Support: Go, Rust, Python, TypeScript, Solidity, Jsonnet, and more
Key Features
- Declarative toolchain management via TOML
- Automatic dependency cell generation for the build system
- Native tool wrappers with auto-sync (
go,cargo,uv) - Modular Nix flake integration
Who Should Use This?
Turnkey is designed for teams who:
- Want reproducible development environments
- Need fast, incremental builds across multiple languages
- Need to manage multiple language toolchains
- Value declarative, version-controlled configuration
Next Steps
- Installation - Set up Turnkey in your project
- Quick Start - Build your first project
Why Turnkey
Modern software development faces a fundamental tension: we want the simplicity of working with familiar tools while also needing the reproducibility and scalability of sophisticated build systems.
Turnkey bridges this gap.
The Problem
Consider a typical development scenario. You have a project that uses Go, some Rust libraries, a Python testing framework, and TypeScript for the frontend. Each language has its own:
- Package manager (go mod, cargo, pip/uv, npm/pnpm)
- Build conventions
- Test runners
- IDE integrations
This works fine for small projects. But as projects grow, you encounter challenges:
- "Works on my machine" - Different developers have different tool versions
- Slow CI/CD - Every change rebuilds everything, even unrelated code
- Dependency hell - Conflicting versions across languages and packages
- AI agent friction - Automated tools struggle with slow, non-incremental builds
The enterprise answer to these problems is typically a monorepo with a sophisticated build system like Bazel or Buck2. But adopting a monorepo means:
- Rewriting all your build logic
- Learning new command-line tools
- Breaking IDE integrations
- Significant upfront investment
The Turnkey Solution
Turnkey takes a different approach: keep your familiar tools working normally while adding build system benefits invisibly.
# These still work exactly as expected
go build ./...
cargo test
pytest
npm run build
# But now you also have Buck2's power when you need it
buck2 build //...
buck2 test //...
The key insight is that most developers don't need to think about the build system most of the time. They want to:
- Write code
- Run tests
- Get fast feedback
Turnkey provides this while maintaining a single source of truth for dependencies and builds that enables advanced features like:
- Hermetic, reproducible builds
- Incremental compilation across languages
- Remote caching and execution
- Atomic changes across the entire codebase
Who Is Turnkey For?
Turnkey is designed for teams that want:
Enterprise-grade infrastructure without abandoning their existing workflows. Your go build still works. Your IDE still works. Your junior developers don't need to learn build system internals to be productive.
A growth path from prototype to production. Start with normal language tooling. Adopt incremental build features as your needs grow. No big-bang rewrites.
AI-friendly development with fast feedback loops. AI coding assistants work better when builds are fast and incremental. Turnkey's caching means AI agents can iterate quickly.
Reproducibility without ceremony. Nix handles tool versioning. The build system handles caching. You focus on writing code.
The Turnkey Philosophy
- Tools should enhance, not replace - Native commands work normally
- Complexity should be opt-in - Start simple, add sophistication as needed
- Reproducibility is non-negotiable - Same inputs always produce same outputs
- Fast feedback enables better code - Incremental builds by default
In the following chapters, we'll explore the core principles that make this possible and how the architecture enables a seamless developer experience.
Core Principles
Turnkey is built on four core principles that guide every design decision. These principles often exist in tension with each other, and Turnkey's value lies in finding the right balance.
1. Native Tool Compatibility
Your existing commands should just work.
When you run go build, it should build your Go code. When you run cargo test, it should test your Rust code. LSP servers should provide autocomplete. IDEs should find definitions. This isn't a compromise - it's a requirement.
How It Works
Turnkey provides transparent wrappers (tw) around native tools that:
- Pass through all commands unchanged by default
- Watch for dependency file changes (go.mod, Cargo.lock, etc.)
- Automatically regenerate build system dependency cells when needed
- Never block or modify the developer's primary workflow
# The 'tw' wrapper is transparent
tw go get github.com/foo/bar # Works exactly like 'go get'
# But also updates build system deps if go.mod changed
# Or use 'go' directly - it still works
go build ./... # Normal Go build, no Buck2 involved
Why This Matters
- Zero learning curve for basic workflows
- IDE integrations continue working - gopls, rust-analyzer, pyright all function normally
- Existing scripts and CI remain valid - no migration required
- Developers stay in their comfort zone while infrastructure improves beneath them
2. Monorepo Benefits Without Monorepo Storage
Get unified versioning without storing the world in your repository.
Traditional monorepos store all code in one repository, enabling atomic changes and unified versioning. But this comes with costs:
- Massive repository size
- Complex code ownership
- Slow git operations
- Storage of third-party code
Turnkey provides the benefits of a monorepo without these costs through virtual cells.
How It Works
your-repo/
├── src/ # Your source code
├── go.mod # Normal Go module
├── Cargo.toml # Normal Cargo workspace
└── .turnkey/
├── godeps/ # Virtual cell: Go dependencies
├── rustdeps/ # Virtual cell: Rust dependencies
└── prelude/ # Virtual cell: Build system prelude
The .turnkey/ directory contains cells - the build system's unit of code organization. These cells are:
- Generated from your lock files (go.sum, Cargo.lock, etc.)
- Deterministically reproducible via Nix
- Treated as source code by the build system (enabling caching and incrementality)
- Never committed to git (they're derived data)
The Result
- Atomic changes across your code and its dependencies
- Unified versioning - one lock file controls one version
- Hermetic builds - Nix ensures reproducibility
- Fast git operations - repository stays small
3. Incremental Build and Test
Only rebuild and retest what actually changed.
Modern CI/CD often wastes enormous resources rebuilding unchanged code. A small typo fix shouldn't trigger a full rebuild of the entire project.
How It Works
The incremental build system tracks fine-grained dependencies between:
- Source files
- Build rules
- Test targets
- Generated artifacts
When a file changes, the build system determines the minimal set of actions needed:
# Edit a single Go file
vim pkg/utils/helper.go
# The build system only rebuilds affected targets
tk build //... # Rebuilds only what depends on helper.go
tk test //... # Runs only tests that might be affected
Combined with remote caching, this means:
- CI builds are fast because most artifacts are cached
- Local builds benefit from CI's cached artifacts
- AI agents can iterate quickly with sub-second feedback
Why This Matters for AI
AI coding assistants (like Claude Code) benefit enormously from fast builds:
- Quick iterations mean more experiments per session
- Fast test feedback enables test-driven development
- Immediate error messages allow rapid course correction
Turnkey's incremental builds make AI-assisted development practical at scale.
4. Continuum of Experience
Scale from prototype to enterprise without rewrites.
Software projects exist on a spectrum:
| Stage | Needs |
|---|---|
| Prototype | Quick iteration, minimal ceremony |
| Startup | Fast CI, some reproducibility |
| Growth | Caching, parallelism, reliability |
| Enterprise | Compliance, governance, audit trails |
Traditional build systems force you to choose: simple but limited, or powerful but complex. Turnkey provides a continuum:
Level 1: Just Use Native Tools
go build ./...
cargo test
pytest
No turnkey involvement. Everything works normally.
Level 2: Add Hermetic Tooling
# toolchain.toml
[toolchains]
go = {}
rust = {}
python = {}
Now your tools are versioned by Nix. "Works on my machine" disappears.
Level 3: Enable Incremental Builds
tk build //...
tk test //...
Get incremental builds and caching. CI becomes faster.
Level 4: Add Remote Caching
Share build artifacts across developers and CI. Builds that took 10 minutes now take 30 seconds.
Level 5: Remote Execution
Distribute builds across a cluster. Massive parallelism. Enterprise scale.
Why This Matters
You don't have to adopt everything at once. Start with Level 1 or 2. Move to higher levels as your needs grow. The underlying infrastructure supports your growth without requiring rewrites.
These four principles - native compatibility, virtual monorepo, incremental builds, and progressive adoption - form the foundation of Turnkey's design. In the next chapter, we'll see how the architecture implements these principles in practice.
Note: Turnkey currently uses Buck2 as its incremental build system. The architecture is designed to potentially support other build systems like Bazel in the future.
Architecture Overview
Turnkey combines three powerful technologies - Nix, an incremental build system, and devenv - into a cohesive developer experience. This chapter explains how these pieces fit together.
Current Implementation: Turnkey uses Buck2 as its incremental build system. The architecture is designed to potentially support other systems like Bazel in the future.
The Three Pillars
┌─────────────────────────────────────────────────────────────┐
│ Developer Experience │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ go build │ │ tk build │ │ IDE / LSP │ │
│ │ cargo test │ │ tk test │ │ Autocomplete │ │
│ │ pytest │ │ tk run │ │ Go to definition │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Turnkey │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ tw wrappers │ │ tk CLI │ │ Dep generators │ │
│ │ Auto-sync │ │ Build wrap │ │ godeps-gen, etc. │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Core Technologies │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Nix │ │Build System │ │ devenv │ │
│ │ Hermetic │ │ Incremental │ │ Shell environment │ │
│ │ packages │ │ builds │ │ configuration │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Nix: Hermetic Package Management
Nix provides reproducible package management. Every tool, compiler, and library has a precise version controlled by the flake.nix and flake.lock files.
What Nix provides:
- Exact versions of go, cargo, python, node, etc.
- System libraries and compilers
- Build tools (buck2 itself)
- Dependency fetching with verified hashes
Key benefit: When you enter the development shell, you have the exact same tools as every other developer and CI system.
Incremental Build System (Buck2)
The build system provides fast, incremental, and correct builds. It tracks dependencies at a fine-grained level and only rebuilds what's necessary.
What the build system provides:
- Dependency tracking between files and targets
- Parallel execution of independent tasks
- Remote caching (share builds across machines)
- Remote execution (distribute builds to a cluster)
Key benefit: After initial setup, builds are dramatically faster because unchanged code isn't rebuilt.
devenv: Developer Shell Configuration
devenv provides a declarative shell environment configured through Nix. It handles:
- Environment variable setup
- Shell hooks and initialization
- Service management (databases, etc.)
- Integration with direnv for automatic activation
Key benefit: Entering a project directory automatically sets up the complete development environment.
The Flow of Data
From Lock Files to Build System Cells
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ go.mod/go.sum │────▶│ godeps-gen │────▶│ go-deps.toml │
│ (native lock) │ │ (generator) │ │ (intermediate) │
└──────────────────┘ └──────────────────┘ └──────────────────┘
│
▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│ .turnkey/godeps │◀────│ Nix │◀────│ go-deps.toml │
│ (Buck2 cell) │ │ (fetcher) │ │ (with hashes) │
└──────────────────┘ └──────────────────┘ └──────────────────┘
- Native lock files (go.sum, Cargo.lock, pnpm-lock.yaml) define exact dependency versions
- Dependency generators (godeps-gen, rustdeps-gen, etc.) parse lock files and output intermediate TOML
- Nix fetches dependencies with verified hashes and creates build-system-compatible cells
- The build system treats these cells as source code, enabling full incrementality
The tw Wrapper Flow
Developer runs: tw go get github.com/foo/bar
│
▼
┌─────────────────┐
│ Snapshot state │ (hash go.mod, go.sum)
└─────────────────┘
│
▼
┌─────────────────┐
│ Run go get │ (native command)
└─────────────────┘
│
▼
┌─────────────────┐
│ Check for diff │ (did lock files change?)
└─────────────────┘
│
┌─────────┴─────────┐
▼ ▼
[No change] [Files changed]
│ │
│ ▼
│ ┌─────────────────┐
│ │ Run godeps-gen │
│ └─────────────────┘
│ │
└───────────────────┘
│
▼
[Done]
The tw wrapper ensures the build system's view of dependencies stays synchronized with native tools, without requiring developer intervention.
Directory Structure
A typical Turnkey-enabled project looks like:
project/
├── .buckconfig → Symlink to generated config (Buck2)
├── .buckroot → Marks project root for build system
├── .envrc → Activates devenv via direnv
├── flake.nix → Nix flake configuration
├── flake.lock → Locked Nix dependencies
├── toolchain.toml → Turnkey toolchain declaration
│
├── src/ → Your source code
│ ├── cmd/
│ ├── pkg/
│ └── rules.star → Build rules
│
├── go.mod → Go module definition
├── go.sum → Go dependency lock
├── go-deps.toml → Generated dependency manifest
│
├── Cargo.toml → Rust workspace definition
├── Cargo.lock → Rust dependency lock
├── rust-deps.toml → Generated dependency manifest
│
└── .turnkey/ → Generated artifacts (gitignored)
├── prelude/ → Build system prelude cell
├── toolchains/ → Toolchain definitions
├── godeps/ → Go dependency cell
├── rustdeps/ → Rust dependency cell
└── sync.toml → Sync configuration
What's Committed to Git
- Source code (
src/) - Native project files (
go.mod,Cargo.toml, etc.) - Lock files (
go.sum,Cargo.lock, etc.) - Turnkey configuration (
toolchain.toml) - Nix configuration (
flake.nix,flake.lock) - Generated dependency manifests (
go-deps.toml, etc.)
What's Generated (Not Committed)
.turnkey/directory (regenerated from lock files).buckconfig(symlinked to Nix store, Buck2-specific)- Build outputs (
buck-out/)
The Toolchain Flow
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ toolchain.toml │────▶│ Registry │────▶│ Nix packages │
│ │ │ (mapping) │ │ │
│ [toolchains] │ │ go = pkgs.go │ │ /nix/store/... │
│ go = {} │ │ rust = pkgs... │ │ │
│ rust = {} │ │ │ │ │
└─────────────────┘ └─────────────────┘ └─────────────────┘
│
▼
┌─────────────────┐ ┌─────────────────┐
│ Buck2 targets │◀────│ mappings │
│ │ │ │
│ toolchains//:go │ │ Generate rules │
│ toolchains//... │ │ from registry │
└─────────────────┘ └─────────────────┘
toolchain.tomldeclares what toolchains you need- The registry maps toolchain names to Nix packages
- mappings translate these into build system toolchain targets
- The build system uses the toolchain targets for builds
Summary
Turnkey's architecture achieves its goals through careful layering:
| Layer | Responsibility | Technology |
|---|---|---|
| Top | Developer UX | Native tools, tw/tk wrappers |
| Middle | Orchestration | Turnkey, dependency generators |
| Bottom | Execution | Nix (packages), build system (builds), devenv (shell) |
Each layer can be understood independently, and the boundaries are clean enough that you can use partial features without understanding the whole system.
For detailed information about specific components, see the reference documentation.
Installation
Prerequisites
Before installing Turnkey, ensure you have:
- Nix with flakes enabled
- direnv (recommended) for automatic environment activation
Enabling Nix Flakes
If you haven't enabled flakes, add to ~/.config/nix/nix.conf:
experimental-features = nix-command flakes
Adding Turnkey to Your Project
New Project
Use the Turnkey template to create a new project:
nix flake init -t github:firefly-engineering/turnkey
Existing Project
Add Turnkey to your flake.nix inputs:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
turnkey.url = "github:firefly-engineering/turnkey";
};
outputs = { self, nixpkgs, turnkey, ... }: {
# Your flake configuration
};
}
Verifying Installation
After setup, enter the development shell:
nix develop
You should see the welcome message and have access to your declared toolchains.
Quick Start
This guide walks you through building your first project with Turnkey.
Create a toolchain.toml
Create a toolchain.toml file in your project root:
[toolchains]
go = {}
This declares that your project needs Go. Buck2 isn't declared: turnkey
ships its own pinned buck2 release, which buck2.enable below adds to the
shell.
Configure Your Flake
Update your flake.nix to use Turnkey:
{
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
turnkey.url = "github:firefly-engineering/turnkey";
devenv.url = "github:cachix/devenv";
};
outputs = inputs@{ turnkey, devenv, ... }:
turnkey.lib.mkFlake { inherit inputs; } {
imports = [
devenv.flakeModule
turnkey.flakeModules.turnkey
];
perSystem = { ... }: {
turnkey.toolchains = {
enable = true;
declarationFiles.default = ./toolchain.toml;
buck2.enable = true;
};
};
};
}
Enter the Shell
nix develop
Build Something
Create a simple Go program and build it with Buck2:
tk build //path/to:target
Next Steps
- Project Setup - Detailed project configuration
- toolchain.toml - Full configuration reference
Project Setup
This guide covers how to create a new Turnkey project or add Turnkey to an existing project.
New Project
Create a new Buck2 project using the Turnkey flake template:
mkdir my-project && cd my-project
nix flake init -t github:firefly-engineering/turnkey
direnv allow # If using direnv
This creates:
flake.nix- Nix flake configuration with Turnkey enabledtoolchain.toml- Toolchain declaration (Go enabled by default).envrc- direnv configuration with symlink sync.gitignore- Ignores Turnkey-managed filesrules.star- Root build file (template)
Existing Project
Add Turnkey to an existing Nix flake project:
1. Add Turnkey Input to flake.nix
{
inputs = {
# ... existing inputs ...
turnkey.url = "github:firefly-engineering/turnkey";
};
outputs = inputs@{ flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } {
imports = [
inputs.devenv.flakeModule
inputs.turnkey.flakeModules.turnkey
];
# ... rest of config ...
perSystem = { pkgs, ... }: {
turnkey = {
enable = true;
declarationFile = ./toolchain.toml;
};
devenv.shells.default = {
turnkey.buck2.enable = true;
};
};
};
}
2. Create toolchain.toml
[toolchains]
go = {}
# Add more as needed: rust, python, cxx
3. Update .gitignore
# Turnkey managed files
.buckconfig
.buckroot
.turnkey/
buck-out/
4. Create .envrc (if using direnv)
Turnkey provides a direnv library that handles all symlink management automatically:
use flake . --no-pure-eval
# Source the turnkey library and activate
source "$TURNKEY_DIRENV_LIB"
use_turnkey
The use_turnkey function handles:
- Buck2 symlink management (
.buckconfig, the prelude and toolchains cells) and the deps cells' directories - Re-evaluating the flake when a deps file changed since its cells were built
watch_filedeclarations for automatic reloads- Optional dependency file regeneration
Then allow it:
direnv allow
Directory Structure
A typical Turnkey project has this structure:
my-project/
├── .buckconfig # Buck2 configuration (generated symlink)
├── .buckroot # Empty file marking project boundary
├── .envrc # direnv configuration
├── .turnkey/ # Generated cells (gitignored)
│ ├── prelude/ # Buck2 prelude
│ ├── toolchains/ # Language toolchains
│ ├── godeps/ # Go dependency cell (if configured)
│ └── rustdeps/ # Rust dependency cell (if configured)
├── flake.nix # Nix flake configuration
├── flake.lock # Locked dependencies
├── toolchain.toml # Toolchain declarations
├── go-deps.toml # Go dependencies (if using Go)
├── rust-deps.toml # Rust dependencies (if using Rust)
└── rules.star # Root build file
Generated Files
When you enter the devenv shell, Turnkey generates:
| File | Description |
|---|---|
.buckconfig | Symlink to Nix-managed Buck2 configuration |
.buckroot | Empty file marking project boundary |
.turnkey/toolchains | Symlink to generated toolchains cell |
.turnkey/godeps | Go dependencies cell, a directory tk materialize maintains (if configured) |
.turnkey/prelude | Symlink to the prelude |
The Prelude
The prelude is always turnkey's: the prelude built with turnkey's pinned buck2 release, with turnkey's patches and extensions applied. It isn't configurable, for the same reason buck2 itself isn't: they are one release (ADR 0002).
If you really need a prelude of your own, prelude.path takes a derivation
or a path. That is off the supported path, and it turns test result caching
off, since turnkey can't know which of that prelude's test rules are
cache-safe:
turnkey.toolchains.buck2.prelude.path = ./my-prelude;
direnv Integration
For automatic environment activation with full Turnkey support, create .envrc:
use flake . --no-pure-eval
source "$TURNKEY_DIRENV_LIB"
use_turnkey
Then allow it:
direnv allow
use_turnkey runs tk sync to regenerate stale dependency files (the
same rules as tk sync everywhere else, from .turnkey/sync.toml), keeps
the cells current and watches the rules' files so that a changed
deps file reloads the shell. When a deps file's content differs from the one
the shell's cells were built from (regenerated by that tk sync, or earlier
by tw or a tk sync of your own), it evaluates the flake again and loads
the .envrc once more, so the cells are rebuilt in the same load. It takes
options:
use_turnkey --skip-regen- Skip dependency file regenerationuse_turnkey --skip-sync- Skip symlink synchronizationuse_turnkey --only-<rule>/--skip-<rule>- Sync only, or all but, the named deps rules (go,rust,pylock,python,javascript,solidity)- Environment variables like
TURNKEY_SKIP_ALL=1(withTURNKEY_ENABLE_<RULE>=1to add rules back) orTURNKEY_SKIP_<RULE>=1
Buck2 Configuration
The .buckconfig is generated automatically. For project-specific settings, create .buckconfig.local:
[build]
# Custom build settings
[project]
# Project-specific settings
Verifying Setup
After entering the shell, verify Buck2 is configured:
# Check toolchains
buck2 targets toolchains//...
# Run Go via toolchain
buck2 run toolchains//:go[go] -- version
# Build a target
buck2 build //...
Common Issues
Symlinks Not Created
If .turnkey/ symlinks aren't created:
- Check that you're using direnv or manually sourcing the environment
- Verify environment variables are set:
echo $TURNKEY_BUCK2_CONFIG echo $TURNKEY_BUCK2_TOOLCHAINS_CELL - Re-allow direnv:
direnv allow
Buck2 Can't Find Cells
If Buck2 reports missing cells:
- Check
.buckconfigis a valid symlink:ls -la .buckconfig - Verify cell paths in
.buckconfigexist:cat .buckconfig - Ensure you've entered the Nix shell:
nix develop
toolchain.toml
The toolchain.toml file declares which toolchains your project needs.
Basic Structure
[toolchains]
go = {}
rust = {}
python = {}
Each key under [toolchains] is a toolchain name that will be resolved from the registry.
buck2 is not declared here. Turnkey ships its own pinned buck2 release to the
shells that have the Buck2 integration, and declaring buck2 or
buck2-toolchain is an error. See
The buck2 version.
Version Pinning
You can pin specific versions when the registry provides multiple versions:
[toolchains]
go = { version = "1.22" } # Pin to Go 1.22
python = { version = "3.11" } # Pin to Python 3.11
rust = {} # Use registry default
If no version is specified, the registry's default version is used.
Available Toolchains
Languages
go- Go compiler and toolsrust- Rust compiler (rustc)cargo- Cargo package managerclippy- Rust linterrustfmt- Rust formatterrust-analyzer- Rust LSP serverpython- Python interpreteruv- Python package managerruff- Python linter and formatternodejs- Node.js runtimetypescript- TypeScript compilerbiome- Fast linter/formatter for JS/TS/JSON
Solidity
solc- Solidity compilerfoundry- Ethereum dev toolkit (forge, cast, anvil)
Other Tools
nix- Nix package managerreindeer- Rust Buck2 target generatorjsonnet- Jsonnet to JSON compilermdbook- Documentation tooltk- Turnkey CLI wrapper for buck2
Internal Tools
Dependency generators (godeps-gen, rustdeps-gen, pydeps-gen, jsdeps-gen, soldeps-gen) are automatically included when their corresponding language is enabled. You don't need to list them in toolchain.toml.
For example, if you have go = {} in your toolchain.toml and buck2.go.enable = true in your flake.nix, godeps-gen will automatically be available in your shell.
Example Configurations
Minimal Go Project
[toolchains]
go = {}
Full-Stack Project
[toolchains]
# Backend
go = {}
python = {}
# Frontend
nodejs = {}
typescript = {}
biome = {}
# Development
nix = {}
Pinned Versions
[toolchains]
go = { version = "1.22" }
python = { version = "3.11" }
nodejs = { version = "20" }
rust = { version = "1.75" }
Custom Registries
The registry mapping toolchain names to packages can be customized in your flake.nix. See Registry Pattern for details on:
- Adding custom toolchains via
registryExtensions - Creating reusable registry overlays
- Multi-version toolchain support
Buck2 Integration
Turnkey provides first-class Buck2 integration with automatic toolchain and dependency cell generation.
Enabling Buck2
In your flake.nix:
turnkey.toolchains = {
enable = true;
declarationFiles.default = ./toolchain.toml;
buck2.enable = true;
};
By default only the default shell gets the Buck2 integration: the pinned
buck2 on its PATH, the prelude, and the generated cells. List other shells
in buck2.shells to give them the integration too:
turnkey.toolchains = {
declarationFiles = {
default = ./toolchain.toml;
ci = ./toolchain.ci.toml;
docs = ./docs/toolchain.toml; # no Buck2 here
};
buck2 = {
enable = true;
shells = [ "default" "ci" ];
};
};
Naming a shell that declarationFiles doesn't define is an error.
The buck2 version
Each turnkey revision ships exactly one buck2 release, together with the
prelude built with it. You don't choose buck2 in toolchain.toml: you get
a newer buck2 by updating your turnkey input (nix flake update turnkey).
Turnkey's test runner speaks buck2's internal test protocol and its prelude
is patched for one prelude revision, so each only works against the release
it was built for
(ADR 0002).
To see which release you have, read turnkey.toolchains.buck2.version, or
look at the shell's welcome message, which ends with (buck2 <version>)
when a welcomeMessage is set.
Migrating from a declared buck2
Earlier turnkey revisions took buck2 from toolchain.toml. Declaring it now
fails evaluation:
error: turnkey: /nix/store/…-source/toolchain.toml declares buck2-toolchain, but turnkey now ships buck2 itself.
To migrate:
- Remove the
buck2orbuck2-toolchainentry from everytoolchain.toml. - Keep
buck2.enable = true;inflake.nix. If a shell other thandefaultused to declare buck2, add its name tobuck2.shells. Before, declaring buck2 is what gave a shell the Buck2 integration. buck2-toolchainalso carriedreindeer. If you use it, declarereindeer = {}intoolchain.toml.
To run a different buck2 anyway, override turnkey's toolbox input with
follows. This is unsupported: test result caching or the prelude may break
against another release.
Generated Cells
When Buck2 integration is enabled, Turnkey generates:
Toolchains Cell
Located at .turnkey/toolchains/, contains toolchain rules for each declared language:
toolchains//:go- Go toolchaintoolchains//:rust- Rust toolchaintoolchains//:python- Python toolchain- etc.
It also holds toolchains//conditions:<os>-<cpu>, one config_setting per
platform in buck2.platforms (the flake's systems by default), combining
its OS and CPU constraints. Rules sync keys a select() on one when deps
differ by CPU within one OS
(Platform-Conditional Deps).
With buck2.go.allowedBuildTags, which also sets .buckconfig's
go.allowed_build_tags, it holds the settings combining the platform's OS
and CPU with each allowed tag, set (linux-x86_64-integration) or unset
(linux-no_integration), for a Go library whose imports depend on a tag.
Prelude Cell
The Buck2 prelude is provided via Nix at .turnkey/prelude/: the prelude built with turnkey's pinned buck2 release, with turnkey's patches and extensions applied.
The prelude and toolchains cells are symlinks into the Nix store. After
either changes, a plain buck2 call needs a tk call or a buck2 kill
first: see Symlinked Cells and Plain buck2.
A Prelude of Your Own
prelude.path replaces turnkey's prelude with a derivation or a path. It is
off the supported path, and it turns test result caching off:
turnkey.toolchains.buck2.prelude.path = ./my-prelude;
Directories Buck2 Doesn't See
The generated .buckconfig's project.ignore lists directories, relative to
the project root, that buck2 skips: //... doesn't load their rules.star
files, and buck2's file watcher drops their events, so they never show up as
File changed: lines. turnkey always ignores two kinds of directory that no
build reads but something writes to all the time:
- the VCS's metadata:
.git,.jj,.hg,.sl; - the devenv and direnv state:
.devenv,.direnv.
ignore adds to them. Use it for trees that are projects of their own, such
as test fixtures whose targets only build in their own checkout:
turnkey.toolchains.buck2.ignore = [ "e2e/fixtures" ];
Buck2 reads project.ignore when its daemon starts, so a change takes effect
once the daemon restarts. .buckconfig is a store symlink, so tk restarts
it on its next command; plain buck2 needs a buck2 kill first
(Symlinked Cells and Plain buck2).
Dependency Cells
Language-specific dependency cells are generated when configured:
godeps//- Go dependencies from go-deps.tomlrustdeps//- Rust dependencies from rust-deps.tomlpydeps//- Python dependencies from python-deps.tomljsdeps//- JavaScript dependencies from js-deps.tomlsoldeps//- Solidity dependencies from solidity-deps.toml
They are write-once directories, which plain buck2 reads without a daemon
restart.
See Managing Dependencies for configuration details.
The .turnkey Directory
Turnkey uses a .turnkey directory in your project root to store build artifacts, caches, and generated cells. This convention provides automatic isolation from language toolchains.
Why .turnkey?
The .turnkey directory serves as the isolation directory for Buck2 builds. By using a dot-prefixed name, we get automatic exclusion from most language toolchains:
| Tool | Behavior | Configuration Needed |
|---|---|---|
| Go | Ignores directories starting with . or _ | None (built-in) |
| Cargo | Doesn't auto-discover crates in dot directories | None (built-in) |
| pytest | Automatically ignores dot directories | None (built-in) |
| Jest | Requires explicit configuration | Yes |
| Vitest | Requires explicit configuration | Yes |
This means Go won't try to compile generated Buck2 cells, Cargo won't discover them as workspace members, and pytest won't scan them for tests.
Directory Structure
.turnkey/
├── books/ # mdbook serve output (gitignored)
├── prelude/ # Symlink to Buck2 prelude derivation
├── toolchains/ # Symlink to generated toolchains cell
├── godeps/ # Real directory: the write-once Go cell (tk materialize)
│ ├── .buckconfig
│ ├── .deps-file-sha256 # the go-deps.toml it was built from
│ ├── _store/<store path name> # one symlink per module, never retargeted
│ └── vendor/<import path>/rules.star # an alias package per Go package
├── godeps.lock # held while tk materialize runs
├── jsdeps/ # Real directory: the write-once JavaScript cell (tk materialize)
│ ├── .buckconfig
│ ├── .deps-file-sha256 # the js-deps.toml it was built from
│ ├── rules.star # an instance per pnpm snapshot, an alias per direct dependency
│ ├── _store/<store path name> # one symlink per package, never retargeted
│ └── vendor/<name>@<version>/rules.star # an alias package per package
├── jsdeps.lock # held while tk materialize runs
├── pydeps/ # Real directory: the write-once Python cell (tk materialize)
│ ├── .buckconfig
│ ├── .deps-file-sha256 # the python-deps.toml it was built from
│ ├── _store/<store path name> # one symlink per distribution, never retargeted
│ └── vendor/<name>/rules.star # an alias package per distribution
├── pydeps.lock # held while tk materialize runs
├── rustdeps/ # Real directory: the write-once Rust cell (tk materialize)
│ ├── .buckconfig
│ ├── .deps-file-sha256 # the rust-deps.toml it was built from
│ ├── _store/<store path name> # one symlink per crate, never retargeted
│ └── vendor/<crate>@<version>/rules.star, vendor/<crate>/rules.star # aliases
├── rustdeps.lock # held while tk materialize runs
├── soldeps/ # Real directory: the write-once Solidity cell (tk materialize)
│ ├── .buckconfig
│ ├── .deps-file-sha256 # the solidity-deps.toml it was built from
│ ├── rules.star # bundle, and an alias per package
│ ├── _store/<store path name> # one symlink per package, never retargeted
│ └── vendor/<name>/rules.star # an alias package per package, beside
│ # links to its files for native forge
├── soldeps.lock # held while tk materialize runs
├── gcroots/godeps, gcroots/jsdeps, gcroots/pydeps, gcroots/rustdeps, gcroots/soldeps # GC roots for the cells' current indexes
├── .cell-targets # the store symlinks' targets, for tk's cell-freshness check
├── .cell-targets.<isolation dir> # the same, for another isolation directory's daemon
├── edits/, patches/ # tk compose's edits and generated patches
└── sync.toml # Symlink to the rules tk sync follows
The prelude and toolchains cells are symlinks to Nix store paths containing
the generated Buck2 cells. The Go, Rust, Python, Solidity and JavaScript
cells are real directories that tk materialize, run by the shell, keeps in
line with the cell index Nix builds: a dependency change rewrites only the
entries for the modules, crates, distributions or packages that changed
(ADR 0004,
ADR 0008,
ADR 0010,
ADR 0011,
ADR 0012).
Don't edit it; the shell rewrites it on every load.
Symlinked Cells and Plain buck2
Two cells are symlinks into the Nix store, repointed when the shell builds a new one:
.turnkey/prelude, the prelude;.turnkey/toolchains, the toolchains cell.
.buckconfig and .turnkey/sync.toml are store symlinks too. They stay
symlinks: they change only when the shell is rebuilt (a turnkey upgrade, a
nix flake update, a toolchain.toml or flake.nix edit), which reloads
it.
The deps cells, rustdeps, godeps, pydeps, jsdeps and soldeps, are
the write-once directories above, which moved off symlinks in
#234: a
dependency change never repoints a link, so any caller, plain buck2
included, reads the new version. A deps cell you set yourself with
buck2.<language>.cell, a derivation without a cell index, is still a
symlink.
A running daemon doesn't notice a repointed symlink. It keeps the build files and sources it already read through the old target, and reads the packages it hadn't loaded from the new one. A build can then mix the old and the new cell, with no error.
tk checks for it. Before each command it syncs for (every buck2
command but the pass-through ones),
tk reads the targets of .buckconfig and of every entry of .turnkey/
that is a symlink into the store, and compares them with those it saved in
.turnkey/.cell-targets. When one was added, removed or repointed, it
prints tk: cell symlink changed, restarting buck2 daemon (unless
--quiet), runs buck2 kill, and saves the new targets. The first run only saves them. The new
daemon starts without the old one's state, so the next build re-runs its
actions.
Each isolation directory has a daemon of its own, so with
--isolation-dir, tk checks against that directory's state file instead,
.turnkey/.cell-targets.<dir> (.cell-targets.turnkey-ci for
tk --isolation-dir=ci), and restarts that directory's daemon:
buck2 --isolation-dir .turnkey-ci kill. A change one daemon was restarted
for still restarts each other one the next time tk runs against it.
The check doesn't cover:
- plain
buck2: a script, a tool that runsbuck2itself, or a shell withTURNKEY_NO_ALIAS=1. The shell'sbuck2alias fortkonly applies to the interactive shell; tk --no-sync, which skips it with the sync;- an isolation directory set only through
BUCK_ISOLATION_DIR: without--isolation-dir,tkrestarts the daemon buck2 picks from the environment, but checks it against.turnkey/.cell-targets, the state of the shell's own.turnkeydaemon. Pass--isolation-dirinstead.
So after the shell is rebuilt, before a plain buck2 call, either:
- run a
tkcommand that syncs, such astk build, which restarts the daemon if a symlink changed; or - run
buck2 kill, with the same--isolation-diras the call.
In turnkey's own repository, these run plain buck2:
- the e2e tests (
e2e/tests/), which callbuck2 build,testandrunin a fixture project's shell; scripts/ci-smoke.sh, whose integration level builds and tests each language's example;check-test-caching(src/cmd/check-test-caching/__main__.py), which runsbuck2 bxl;- the
starlark-lintgit hook,buck2 --isolation-dir .turnkey-lint starlark lint. It has a daemon of its own, whichtknever restarts: after a turnkey upgrade, runbuck2 --isolation-dir .turnkey-lint kill.
Buck2 Configuration
The .buckconfig sets the isolation directory:
[buck2]
isolation_dir = .turnkey
This tells Buck2 to store all build outputs under .turnkey/buck-out/ instead of the default buck-out/.
The tk Command
The tk command wraps buck2 and automatically translates the --isolation-dir flag to use .turnkey-prefixed directories:
# These are equivalent:
tk --isolation-dir=foo build //...
buck2 --isolation-dir=.turnkey-foo build //...
This allows multiple isolated builds while maintaining the dot-prefix convention.
JavaScript/TypeScript Configuration
Unlike Go, Cargo, and pytest, JavaScript test runners need explicit configuration to ignore dot directories.
Jest
Add to your jest.config.js:
module.exports = {
testPathIgnorePatterns: [
'/node_modules/',
'/buck-out/',
'/\\.' // Ignore all dot-prefixed directories
],
};
Or in package.json:
{
"jest": {
"testPathIgnorePatterns": [
"/node_modules/",
"/buck-out/",
"/\\."
]
}
}
Vitest
Add to your vitest.config.ts:
import { defineConfig } from 'vitest/config';
export default defineConfig({
test: {
exclude: [
'**/node_modules/**',
'**/buck-out/**',
'**/.*/**' // Ignore all dot-prefixed directories
],
},
});
Migration Notes
If you're migrating from a project that used buck-out/ directly:
-
One-time cache invalidation: Buck2 caches are stored per isolation directory. Switching to
.turnkeymeans a clean rebuild on first run. -
Update .gitignore: Ensure
.turnkey/is in your.gitignore:.turnkey/ -
Update CI scripts: If CI scripts reference
buck-out/, update them to.turnkey/buck-out/.
Multiple Isolation Directories
For advanced use cases (parallel builds, different configurations), you can use multiple isolation directories:
# Development build
tk build //...
# Release build with different isolation
tk --isolation-dir=release build //...
# Creates .turnkey-release/
# CI build
tk --isolation-dir=ci build //...
# Creates .turnkey-ci/
Each isolation directory maintains its own:
- Buck2 daemon
- Build cache
- Output artifacts
This is useful for:
- Running multiple Buck2 daemons simultaneously
- Keeping CI caches separate from local development
- Testing different build configurations
Shell Environment
Turnkey configures your development shell with all declared toolchains.
Environment Variables
When entering the shell, Turnkey sets:
PATH- Includes all toolchain binariesTURNKEY_DIRENV_LIB- Path to direnv integration library
direnv Integration
For automatic shell activation, use direnv with .envrc:
use flake
Shell Entry Hooks
Turnkey performs these actions on shell entry:
- Symlinks
.turnkey/preludeto the prelude cell - Symlinks
.turnkey/toolchainsto the generated toolchains - Brings the dependency cells' directories in line with their cell indexes (
tk materialize) - Displays welcome message (if configured)
Verbose Mode
For debugging, set TURNKEY_VERBOSE=1:
TURNKEY_VERBOSE=1 nix develop
Multiple Shells
You can define multiple shells with different toolchains:
turnkey.toolchains.declarationFiles = {
default = ./toolchain.toml;
ci = ./toolchain.ci.toml;
};
Access with:
nix develop .#ci
IDE Integration
This guide explains how to configure your IDE to work seamlessly with Turnkey's automatic dependency synchronization.
Overview
Turnkey can automatically update rules.star files when you modify source code imports. While this happens automatically when running tk build, you can also configure your IDE to trigger sync on file save for immediate feedback.
VS Code
Run on Save Extension
Install the Run on Save extension, then add to your workspace .vscode/settings.json:
{
"emeraldwalk.runonsave": {
"commands": [
{
"match": "\\.(go|rs|py|ts|tsx|sol)$",
"cmd": "tk rules sync --quiet ${fileDirname}"
}
]
}
}
This runs tk rules sync on the directory containing the modified file whenever you save a source file.
Task-based Approach
Alternatively, create a VS Code task in .vscode/tasks.json:
{
"version": "2.0.0",
"tasks": [
{
"label": "Sync rules.star",
"type": "shell",
"command": "tk rules sync",
"presentation": {
"reveal": "silent",
"panel": "shared"
},
"problemMatcher": []
}
]
}
Then bind it to a keyboard shortcut in keybindings.json:
{
"key": "ctrl+shift+s",
"command": "workbench.action.tasks.runTask",
"args": "Sync rules.star"
}
JetBrains IDEs (IntelliJ, GoLand, PyCharm, etc.)
File Watchers
-
Go to Settings > Tools > File Watchers
-
Click + to add a new watcher
-
Configure:
- Name:
Turnkey Rules Sync - File type:
Go files(or your language) - Scope:
Project Files - Program:
tk - Arguments:
rules sync --quiet $FileDir$ - Output paths to refresh:
$FileDir$/rules.star - Working directory:
$ProjectFileDir$
- Name:
-
Under Advanced Options:
- Check: "Trigger the watcher on external changes"
- Uncheck: "Auto-save edited files to trigger the watcher"
External Tools
Alternatively, set up an external tool:
-
Go to Settings > Tools > External Tools
-
Click + to add:
- Name:
Sync rules.star - Program:
tk - Arguments:
rules sync - Working directory:
$ProjectFileDir$
- Name:
-
Assign a keyboard shortcut in Keymap settings
Neovim
Add to your Neovim configuration:
-- Auto-run tk rules sync on save for supported file types
vim.api.nvim_create_autocmd("BufWritePost", {
pattern = { "*.go", "*.rs", "*.py", "*.ts", "*.tsx", "*.sol" },
callback = function()
local file_dir = vim.fn.expand("%:p:h")
vim.fn.jobstart({ "tk", "rules", "sync", "--quiet", file_dir }, {
on_exit = function(_, code)
if code ~= 0 then
vim.notify("tk rules sync failed", vim.log.levels.WARN)
end
end,
})
end,
})
Emacs
Add to your Emacs configuration:
(defun turnkey-sync-rules ()
"Run tk rules sync on the current file's directory."
(when (and buffer-file-name
(string-match-p "\\.\\(go\\|rs\\|py\\|ts\\|tsx\\|sol\\)$" buffer-file-name))
(let ((default-directory (file-name-directory buffer-file-name)))
(start-process "tk-rules-sync" nil "tk" "rules" "sync" "--quiet" "."))))
(add-hook 'after-save-hook #'turnkey-sync-rules)
Configuration Options
Module Options
Rules sync is configured through turnkey's Buck2 options in your
flake.nix, which generate the [rules] section of .turnkey/sync.toml
(a generated file: don't edit it):
turnkey.toolchains.buck2.rules = {
enabled = true; # Enable rules.star sync (default: false)
autoSync = true; # Auto-sync before tk build (default: true)
strict = false; # Fail if rules would change - for CI (default: false)
};
Sync finds each language's internal targets from its own manifest
(go.mod, Cargo.toml, the uv workspace) and uses turnkey's deps cells
(godeps, rustdeps, pydeps, jsdeps, soldeps), through the deps
file each is built from (the language's depsFile). Both reach sync
through the [[languages]] of .turnkey/sync.toml. The platforms it
resolves deps for come from buck2.platforms (see
Platform-Conditional Deps).
Command Line Options
tk rules sync # Sync only stale files (git-based detection)
tk rules sync --force # Force sync all files
tk rules sync --verbose # Show detailed output
tk rules sync --dry-run # Show what would change without writing
tk rules check # Check every rules.star file (exit 1 if any is stale)
Staleness Detection
tk rules sync and the sync tk runs before buck2 commands skip work that can't have changed:
- Git-based: only directories with uncommitted source file changes are considered
- Mtime-based: within those, a
rules.starnewer than every source file next to it is skipped
--force (or --all) turns both off. This means tk rules sync is nearly instant in most cases, making it suitable for on-save hooks.
tk rules check uses neither: it always checks every rules.star. Once a stale rules.star is committed, git reports no change for it and nothing makes its sources newer, so a filtered check would pass it forever.
Preservation Markers
If you have manual dependencies that shouldn't be auto-managed, use preservation markers:
go_binary(
name = "my-app",
srcs = ["main.go"],
deps = [
# turnkey:auto-start
"godeps//vendor/github.com/google/uuid:uuid",
# turnkey:auto-end
# turnkey:preserve-start
# Manual override for special case
"//special:dep",
# turnkey:preserve-end
],
)
Dependencies between preserve-start and preserve-end markers are never modified by sync.
Opting a Target Out
To keep sync away from one target entirely, for example to work around a
problem, put a # turnkey:no-sync comment on its own line right before the
rule:
# Links a hand-built native library sync knows nothing about
# turnkey:no-sync
rust_library(
name = "my-lib-native",
deps = _COMMON_DEPS + ["//third-party/native:lib"],
)
Sync never changes an opted-out target. tk rules sync -v and
tk rules check -v list them as OPTED OUT:.
Platform-Conditional Deps
Some deps are only needed on some platforms. Sync resolves every target's
deps on each platform the project builds for, so what it writes is the same
whichever machine runs it. The platforms are buck2.platforms, the flake's
systems by default:
turnkey.toolchains.buck2.platforms = [ "x86_64-linux" "aarch64-darwin" ];
They reach sync through the [conditions] section of .turnkey/sync.toml,
in Buck2's names:
[conditions]
settings = "toolchains//conditions"
[[conditions.platforms]]
os = "linux"
cpu = "x86_64"
Deps every platform needs are written as a plain list. The others are
written as a select() after it:
rust_library(
name = "my-lib",
deps = [
# turnkey:auto-start
"rustdeps//vendor/libc:libc",
# turnkey:auto-end
] + select({
"config//os:linux": ["rustdeps//vendor/fuser:fuser"],
"config//os:macos": [],
}),
)
- A key is the smallest one that says exactly where the deps apply:
config//os:<os>when they differ only by OS (orconfig//cpu:<cpu>by CPU alone), otherwise one of the toolchains cell'sconfig_settings combining both,toolchains//conditions:<os>-<cpu>, one per platform. Go build tags are dimensions too (see Go below), combined the same way. - Every platform gets a branch, empty if it needs nothing more, and there is
no
DEFAULT: building for a platform that isn't listed fails instead of silently missing deps. - Sync reads this form back and owns all of it. The
turnkey:autoandturnkey:preservemarkers apply to the plain list only. A change to one branch rewrites only that branch. - A target whose deps don't depend on the platform keeps a plain list.
A select() sync can't read (its keys aren't config//os:*,
config//cpu:*, toolchains//conditions:* or DEFAULT, or its values
aren't lists of labels) makes the target unreadable, like any other
expression.
Where Deps Come From
- Rust: the crate's
Cargo.toml, not its sources.[dependencies]go to library and binary targets; test targets get[dependencies]plus[dev-dependencies].workspace = trueentries are resolved against the rootCargo.toml, a workspace member maps to its own target, and any other crate maps torustdeps//vendor/<package>:<package>(a renamed dependency maps to itspackage). An existing label that pins a version (rustdeps//vendor/tokio@1.50.0:tokio) satisfies the unversioned one. A target-specific table ([target.'cfg(...)'.dependencies], or a target triple) applies on the platforms its spec holds on, so its deps are platform-conditional: a dep every platform gets is a plain dep, one no platform gets is dropped.cfg()supportstarget_os,target_family(unix),target_arch,target_pointer_width,target_env,target_vendor,target_endianandall/any/not. Build dependencies are not synced: sync reports them and leaves any existing dep on them alone. See Rust Features for features, optional dependencies and dependencies on a member's variant. - Go: the imports
go listreports, on every platform: it runs once per platform withGOOS,GOARCHandCGO_ENABLED=1set, so a_linux.gofile's imports become aconfig//os:linuxbranch whichever machine runs sync. A binary or a test is built with itsbuild_tags, taken literally (none is a plaingo build). A library gets its tags from the configuration, through the prelude's transition: each tag inbuck2.go.allowedBuildTagsthat its build constraints use is an on/off dimension, and imports that depend on one become aselect()onprelude//go/tags/constraints:<tag>[set]/[unset], or on a toolchains cell setting combining it with the platform (toolchains//conditions:linux-<tag>,...-no_<tag>). Test targets get_test.goimports, external test packages' (XTestImports) included. - Python: the imports found in the sources. An import of a package that a
uv workspace member provides (
turnkey.cfg,from turnkey import cfg) maps to that member's target. The packages come from the members listed in the rootpyproject.toml's[tool.uv.workspace]and their source layout, so a downstream namespace such asacme.*works the same way. Any other import maps topydeps. - TypeScript: the npm packages imported by the sources, written to the
target's
npm_depsas the jsdeps cell'sjsdeps//:<npm name>aliases (jsdeps//:@types/lodash), plus each one's@types/...package when it is a direct dependency too. Only the rootpackage.json's dependencies (js-deps.toml's[direct]) map: an import of anything else is unmapped. The target'sdeps(other TypeScript targets) are not synced. - Other languages: the imports found in the sources, mapped to targets.
Sync never removes a dep it can't account for:
- If a target has imports sync can't map to a target (reported as
unmapped import), its deps are incomplete, so sync adds what it resolved but removes nothing, and prints aKEPT:line naming the deps it kept and why. - A target whose
depsis an expression (_DEPS,_COMMON_DEPS + [...]) rather than a list of labels is not synced. Unless# turnkey:no-syncopts it out,tk rules syncandtk rules checkreport it asUNREADABLE:, and it makestk rules checkand strict mode fail: write its deps as a list of labels, or opt it out. (The sync before a build doesn't report it.) - Deps are only added or removed, never reordered.
Rust Features
A Rust target builds what Cargo would, and sync keeps it that way: it
writes the target's features (the literal list the prelude passes to
rustc, one --cfg feature="..." each) as well as its deps.
-
A primary target, one that sets neither of the attributes below, builds what
cargo build -p <crate>builds: the crate'sdefaultfeatures, expanded. An optional dependency is a dep only when an enabled feature activates it (dep:x, an implicit feature,x/feat). -
A variant asks for features in Cargo's terms, on two attributes of turnkey's prelude Rust rules that rustc never sees:
rust_library( name = "composition-full", crate = "composition", cargo_features = ["watcher"] + select({ "config//os:linux": ["fuse"], "config//os:macos": ["fuse-t"], }), # default_features = False, # as Cargo's default-features )Sync expands the request as Cargo would for a dependency asking for those features (
dep:x,x/feat, weakx?/feat, feature-to-feature, anddefaultunlessdefault_features = False), and writes thefeaturesanddepsit gives, per platform when the request is aselect(). -
featuresis literal:defaultis written only when listed, and Buck2 adds nothing implicit. -
A dependency on a workspace member that asks for features (its own
features = [...]with the[workspace.dependencies]entry's, those its crate's features forward to it, and the member's defaults unlessdefault-features = false) maps to the member'srust_librarywhose request enables exactly the same features, per platform. If none or several do, the dependency is reported as an unmapped import naming the features it needs, and nothing is removed.
A crate that doesn't build under Buck2 is a bug to fix in the rustdeps cell, not a reason to make a target a variant.
rust-analyzer sees a select()'d features through the host's branch.
Troubleshooting
Sync not running
- Ensure
deps-extractis in your PATH (built withcargo install --path src/rust/deps-extract) - Check that
[rules] enabled = truein.turnkey/sync.toml - Verify the file type is supported (Go, Rust, Python, TypeScript, Solidity)
Sync too slow
- Use the default staleness detection (don't use
--forcein on-save hooks) - Target a specific directory:
tk rules sync src/cmd/myapp
Wrong dependencies detected
- Check your
*-deps.tomlfiles are up to date (runtk sync) - Verify internal prefix configuration in sync.toml
- Run
tk rules sync --verboseto see what's being detected
FUSE Composition Layer
The FUSE composition layer provides a unified filesystem view of your repository and its dependencies at a fixed mount location. This enables:
- Predictable paths for remote cache compatibility
- Transparent editing of external dependencies
- Automatic consistency management during updates
Quick Start
Manual (ad-hoc)
# Start the daemon for a single repo
turnkey-composed start --mount-point ~/firefly/turnkey --repo-root . --backend fuse
# Work from the mount point
cd ~/firefly/turnkey
buck2 build root//...
# Stop
turnkey-composed stop
As a service (recommended)
# Install the service (runs on login)
turnkey-composed install --start
# Edit the config to declare your mounts
vim ~/.config/turnkey/composed.toml
With home-manager (declarative)
{
imports = [ turnkey.homeManagerModules.turnkey-composed ];
services.turnkey-composed = {
enable = true;
package = turnkey.packages.${system}.turnkey-composed;
mounts = {
myproject = {
repo = "/Users/me/src/myproject";
mountPoint = "/firefly/myproject";
};
};
};
}
Prerequisites
Linux
# Verify FUSE is available
ls /dev/fuse
# If missing, install fuse3
sudo apt install fuse3 # Debian/Ubuntu
sudo dnf install fuse3 # Fedora
macOS
Install FUSE-T (no kernel extension, works on Apple Silicon):
brew install macos-fuse-t/homebrew-cask/fuse-t
Mount points under /: macOS root is read-only. The daemon
automatically manages /etc/synthetic.conf entries and activates them
via apfs.util -t when a mount point like /firefly/turnkey is
requested. This requires sudo (the daemon prompts when needed).
For paths under ~ (e.g., ~/firefly/turnkey), no special setup is
needed.
Service Configuration
The service reads ~/.config/turnkey/composed.toml:
# Mount a project
[[mounts]]
repo = "/Users/me/src/myproject"
mount_point = "/firefly/myproject"
# Mount another project
[[mounts]]
repo = "/Users/me/src/other-project"
mount_point = "/firefly/other"
backend = "fuse" # Optional: "auto" (default), "fuse", or "symlink"
The daemon watches this file for changes. When you add a new [[mounts]]
entry, the daemon picks it up and mounts it automatically — no restart
needed.
Home-Manager Module
The declarative alternative to editing the TOML file directly:
{
imports = [ turnkey.homeManagerModules.turnkey-composed ];
services.turnkey-composed = {
enable = true;
package = turnkey.packages.${system}.turnkey-composed;
mounts = {
myproject = {
repo = "/Users/me/src/myproject";
mountPoint = "/firefly/myproject";
};
other = {
repo = "/Users/me/src/other";
mountPoint = "/firefly/other";
backend = "fuse"; # Optional
};
};
};
}
This generates the config file and manages the launchd agent (macOS) or systemd user service (Linux).
Service Management
# Install and start the service
turnkey-composed install --start
# Uninstall the service
turnkey-composed uninstall
# The service runs `turnkey-composed serve` which:
# - Reads ~/.config/turnkey/composed.toml
# - Builds cells via nix for each repo
# - Mounts all entries
# - Watches for config and manifest changes
A buck2 daemon keeps the mount it started on. When turnkey-composed
mounts with FUSE, it kills the buck2 daemons of the projects inside the
mount point, so a restarted service needs no buck2 kill: the next buck2
command starts a fresh daemon.
How Cell Discovery Works
On startup, turnkey-composed:
- Runs
nix evalto list*-cellpackages from each repo's flake - Runs
nix buildto build all cells in a single invocation (~3-4s if cached) - Uses the Nix store paths to populate
external/in the FUSE mount
Cells are always built from the current flake state. The daemon watches
manifest files (go-deps.toml, rust-deps.toml, etc.) and rebuilds
cells automatically when they change.
Mount Structure
/firefly/myproject/
├── .buckconfig # Virtual - generated by layout
├── .buckroot # Virtual - marks Buck2 root
├── root/ # Pass-through to your repository
│ ├── src/
│ ├── docs/
│ ├── flake.nix
│ └── ...
└── external/ # Dependency cells (from Nix store)
├── godeps/
├── rustdeps/
├── prelude/
├── toolchains/
└── ...
Buck2 runs from the mount root. Source targets use the root// cell
prefix: buck2 build root//src/cmd/tk:tk.
CLI Reference
Single Mount
# Start (foreground)
turnkey-composed start --mount-point <path> --repo-root <path> [--backend fuse|symlink|auto]
# With explicit config file
turnkey-composed start --config <path>
Service Mode
# Run as a service (reads ~/.config/turnkey/composed.toml)
turnkey-composed serve [--config <path>]
# Install/uninstall the system service
turnkey-composed install [--start]
turnkey-composed uninstall
Control
turnkey-composed status # Check daemon status
turnkey-composed refresh # Trigger manual cell rebuild
turnkey-composed stop # Stop the daemon
Platform Notes
Linux
Uses native FUSE via /dev/fuse with the fuser Rust crate. Best
performance.
macOS
Uses FUSE-T with direct C FFI bindings to libfuse3. FUSE-T translates FUSE operations to NFS internally. No kernel extension required.
The daemon handles synthetic firmlinks automatically for mount points
under / (manages /etc/synthetic.conf and runs apfs.util -t).
Symlinks (CI / Fallback)
Fastest for CI. No daemon needed. Automatically selected when FUSE is unavailable.
Integration with IDEs
VS Code / Cursor
{
"go.goroot": "/firefly/myproject/root",
"rust-analyzer.linkedProjects": ["/firefly/myproject/root/Cargo.toml"]
}
IntelliJ / GoLand
Set the project root to the FUSE mount point for consistent path resolution.
Building Projects
Turnkey integrates with Buck2 for building projects.
The tk Command
Use tk instead of buck2 directly. It provides:
- Automatic dependency sync before builds
- A daemon restart when a symlinked cell changed (Symlinked Cells and Plain buck2)
- Consistent behavior across the team
tk build //path/to:target
Common Build Commands
# Build a specific target
tk build //src/examples/go-hello:go-hello
# Build all targets
tk build //...
# Build with verbose output
tk build //... -v
# Build in release mode
tk build //... -c release
Build Outputs
Build outputs are placed in buck-out/.turnkey/:
buck-out/
└── .turnkey/
├── gen/
│ └── root/
│ └── path/to/target/
└── tmp/
└── ...
Why .turnkey? The isolation directory starts with a dot so that language tools ignore it:
- Go skips directories starting with
.when scanning for packages - Cargo ignores dot-directories
- pytest ignores dot-directories by default
This prevents errors like Go trying to parse generated .go files in build outputs, or pytest collecting test files from there.
To find the output path for a specific target:
tk build //path/to:target --show-output
Skipping Sync
If you know dependencies haven't changed:
tk --no-sync build //...
Troubleshooting
Missing Toolchain
If you see "toolchain not found", ensure:
- The toolchain is declared in
toolchain.toml - You've re-entered the shell after adding it
Stale Dependencies
If builds fail with missing dependencies:
tk sync
tk build //...
Running Tests
Turnkey supports running tests via Buck2.
Test Commands
# Run tests for a specific target
tk test //path/to:target-test
# Run all tests
tk test //...
# Run tests matching a pattern
tk test //src/examples/...
tk test reuses the recorded result of a test whose inputs haven't changed
instead of running it again; see Test Result Caching.
Use tk --rerun test to run everything.
Language-Specific Tests
Go Tests
tk test //src/go/pkg/mypackage:mypackage_test
Rust Tests
tk test //src/rust/mycrate:mycrate-test
Python Tests
tk test //src/python/mymodule:test
Test Output
Only tests that didn't pass are listed, each with its stdout and stderr; the summary line counts the rest. To list passing tests with their output too:
tk test //... -- --print-passing-details
Filtering Tests
Arguments after -- go to the test runner, not the test binary. Pass them
on to the binary with --test-arg, which takes every argument after it, so
it comes last:
# Run specific test function (Go)
tk test //pkg:pkg_test -- --test-arg -test.run=TestSpecificFunction
# Run specific test (Rust)
tk test //crate:crate-test -- --test-arg specific_test_name
A filtered run has its own result key, so it doesn't reuse the unfiltered run's result.
Continuous Testing
For development, use Buck2's file watching:
tk test //path/to:target-test --watch
Test Result Caching
tk test reuses a test's recorded result when nothing it depends on has
changed, instead of running it again. buck2 already caches build actions;
test result caching extends that to test runs. It is on by default.
$ tk test //src/...
Tests finished: Pass 42. Fail 0. Timeout 0. Fatal 0. Skip 0. Omit 0. Infra Failure 0. Build failure 0
38 recorded (reused without running)
A reused result is a hit. The last line counts the hits. Exit codes are
the same as when every test runs: 0 if all tests pass, 32 if any fails.
Passing tests, hits included, aren't listed: only tests that didn't pass are,
with their output. To list every test with its output, pass
--print-passing-details to the test runner:
$ tk test //src/... -- --print-passing-details
✓ Pass: root//src/rust/starlark-parse:starlark-parse-test
recorded: reused the result of an earlier run with the same inputs
---- STDOUT ----
...
A hit is then marked recorded under the test's line, shows no duration (the
test didn't run), and prints the output of the run that recorded it.
When a result is reused
A result is reused only when its result key is unchanged. The key covers everything the test can see:
- its inputs (sources, dependencies, data files, the toolchain);
- its command line, including arguments after
--such as--test-arg(so a filtered run has its own key); - its declared environment, including
--env; - its timeout, working directory and platform;
- the buck2 release and the caching tool's version.
It works across buck2 daemon restarts, tk clean, and other checkouts of the
same revision on the same machine (jj workspaces, git worktrees).
Only passes are recorded. A test that failed, timed out or crashed always runs again.
Caching applies only under tk test. A plain buck2 test neither reuses nor
records results.
Forcing a re-run
tk --rerun test //src/...
runs every matched test, and records the fresh passes, which replace the old
ones. Like --no-sync, the flag goes before the subcommand.
Which targets are cached
Every target of a cache-safe rule is: turnkey's rules and the rust, go and
python test rules. With caching on, they carry the label turnkey-cacheable,
which buck2 uquery shows. It is added by the rules, never by hand.
Opting a target out
Label a target no-test-cache to always run it and never record it, for
example a test that talks to the network or depends on the time:
go_test(
name = "integration_test",
srcs = ["integration_test.go"],
# Talks to a staging server.
labels = ["no-test-cache"],
)
For solidity_test, the label also switches fuzzing back to a random seed;
cached Solidity tests fuzz with a seed derived from the target's label. A
solidity_test with fork_url is never cached, because it reads chain state
over the network.
What each language does
A test is only cached when its rule can't read anything outside its result key. For cached tests, turnkey:
- sets
PATHto Nix store paths only (bash, coreutils, diffutils), andHOMEto/homeless-shelter, a directory that doesn't exist. A test that needs a writable directory should use$TMPDIR; - runs the command with project-relative paths, from the project root.
| Language | Notes |
|---|---|
| Rust | No other changes. |
| Go | Tests that read fixtures must declare them as resources. |
| Python | Runs the toolchain's interpreter by store path, with PYTHONDONTWRITEBYTECODE=1. |
| Jsonnet | import only resolves from the test's declared sources and dependencies. |
| Solidity | forge uses the toolchain's solc, offline, with dependencies from the soldeps cell. |
Configuration
Test result caching is configured in your flake, under the Buck2 integration:
turnkey.toolchains.buck2.testCache = {
enable = true; # default; false runs tests under buck2's bundled runner
endpoint = null; # default: a cache on this machine, managed by tk
tls = true; # only used with a remote endpoint
};
The local cache
Recorded results live in one store per user per machine, shared by every checkout and every turnkey repo:
| Platform | Location |
|---|---|
| macOS | ~/Library/Caches/turnkey/test-results/ |
| Linux | ~/.cache/turnkey/test-results/ |
tk test starts the cache server (bazel-remote) on demand, in the
background. It stops by itself after 24 hours without use, and the next
tk test starts it again; recorded results stay in the store. Its log is
server.log in the store. The store is limited to 5 GiB; the least recently
used results are dropped first.
If the cache can't be reached or started, tk test runs the tests uncached
and prints one line:
tk: running tests without the test result cache: <reason>
Per-user settings
These are set in your environment, not in the repo:
| Variable | Effect |
|---|---|
TURNKEY_CACHE_DIR | Keep turnkey's caches here instead of the platform cache directory. |
TURNKEY_TEST_CACHE_SIZE_GIB | Store size limit in GiB (default 5). |
TURNKEY_TEST_CACHE_PORT | Port of the local cache server (default 47301). Read when the dev shell is evaluated, so reload the shell after changing it. |
A remote cache
Set endpoint to use a shared Remote Execution API cache instead of the
local one:
turnkey.toolchains.buck2.testCache.endpoint = "grpc://cache.example.com:443";
tk test then starts no local cache, and results found there are marked
recorded, remote. Nothing is recorded to a remote cache yet: who may write
to a shared cache hasn't been decided. tk can't check a remote cache
before running, so if it is unreachable, buck2 retries for about 45 seconds
and then runs the tests locally, uncached.
In the event log
For every hit, the test result in buck2's event log (buck2 log show)
carries a JSON record in its msg field:
{"turnkey_test_cache": {"hit": true, "origin": "local", "original_duration_us": 475340}}
origin is local or remote, and original_duration_us is how long the
recorded run took. The test's TestEnd event reports the execution as a
RemoteCommand with cache_hit: true.
Caching your own test rules
A custom test rule opts in by passing the arguments it gives
ExternalRunnerTestInfo through test_caching_kwargs:
load("@prelude//test_caching:test_caching.bzl", "test_caching_kwargs")
def _my_test_impl(ctx):
command = cmd_args(ctx.attrs.runner[RunInfo], ctx.attrs.src)
return [
DefaultInfo(),
ExternalRunnerTestInfo(**test_caching_kwargs(
{
"type": "my_language",
"command": [command],
"env": ctx.attrs.env,
"labels": ctx.attrs.labels,
},
# Store-path bin directories the test needs beyond bash,
# coreutils and diffutils.
extra_path = [],
# Variables only cached runs need.
extra_env = {},
)),
]
When caching is off, the arguments come back unchanged. When it is on, the
helper also gives the test an executor that reads recorded results. A rule
that supports remote execution passes its re_executors as well, and a test
that upstream runs remotely keeps its executor.
Only opt a rule in
when its tests can't read anything that isn't in their result key: every
file they read must be a declared input, and every tool must come from a
store path.
Managing Dependencies
This guide covers how external dependencies are managed in Turnkey projects.
Core Principles
1. No In-Repo Vendoring
Dependencies are never vendored into the repository. All dependency sources live in the Nix store.
- No
vendor/directories committed to git - No
node_modules/,__pycache__/, or similar cached dependencies - The repository contains only source code and dependency declarations
2. Language-Native Declarations Are the Source of Truth
Each language has its own dependency declaration format. These are the sole source of truth for what dependencies are needed:
| Language | Declaration Files |
|---|---|
| Go | go.mod, go.sum |
| Rust | Cargo.toml, Cargo.lock |
| Python | pyproject.toml, uv.lock |
These files define the dependency graph at the module level (not package/subpackage level).
3. Per-Module Fetching with Deterministic Hashes
Dependencies are fetched individually by Nix, each with its own content hash:
go.mod/go.sum → godeps-gen → go-deps.toml → Nix fetches each module
The intermediate TOML file (go-deps.toml, rust-deps.toml, etc.) contains:
- Module/crate/package identifiers
- Versions (from lock file)
- Nix-compatible SRI hashes (from prefetching)
4. Dependency Cells for Buck2
Dependencies are assembled into Buck2 cells by Nix:
go-deps.toml → one Nix package per module → .turnkey/godeps/ (tk materialize)
The cell contains:
- Fetched source files for each dependency
- Generated rules.star files for Buck2 to consume
- Any scaffolding needed by build tools (e.g.,
modules.txtfor Go)
Data Flow
┌─────────────────────────────────────────────────────────────────────────┐
│ Source of Truth │
│ │
│ go.mod / go.sum Cargo.toml / Cargo.lock pyproject.toml │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Hash Generation Tools │
│ │
│ godeps-gen rustdeps-gen pydeps-gen │
│ │
│ Reads dependency declaration, fetches each module via nix-prefetch-* │
│ Outputs TOML with per-module SRI hashes │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Dependency TOML Files │
│ │
│ go-deps.toml rust-deps.toml python-deps.toml │
│ │
│ [deps."github.com/foo/bar"] │
│ version = "v1.2.3" │
│ hash = "sha256-..." │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Nix Cell Builders │
│ │
│ nix/lib/deps-cell/adapters/{go,rust,python,javascript,solidity}.nix │
│ │
│ - Reads TOML, fetches each module via fetchFromGitHub/fetchurl │
│ - Assembles into directory structure │
│ - Generates rules.star files │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Buck2 Cells (in .turnkey/) │
│ │
│ .turnkey/godeps/ .turnkey/rustdeps/ .turnkey/pydeps/ │
│ (directories of links into the Nix store) │
│ │
│ Contains: source files, rules.star files, cell config │
└─────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────┐
│ Buck2 Build │
│ │
│ buck2 build //my/package:target │
│ │
│ References deps as: godeps//vendor/github.com/foo/bar:bar │
│ All sources already in Nix store - no network access needed │
└─────────────────────────────────────────────────────────────────────────┘
Auto-Sync with Wrapped Tools
When using go, cargo, or uv in a Turnkey shell, the tools are
transparently wrapped to trigger automatic dependency synchronization when
dependency files change.
# These trigger auto-sync when dependency files change
go get github.com/some/package
cargo add serde
uv add requests
How Auto-Sync Works
- The wrapper captures a hash of dependency files before running the command
- The actual tool runs (e.g.,
go get) - After completion, the wrapper checks if dependency files changed
- If changed,
tk syncis triggered automatically
Verbose Mode
Use verbose mode to see what the wrapper is doing:
tw -v go get github.com/some/package
Manual Sync
Force a full dependency sync with:
tk sync
Or sync specific languages:
tk sync --go
tk sync --rust
tk sync --python
Go Dependencies
Configuration
turnkey.toolchains.buck2.go = {
enable = true;
depsFile = ./go-deps.toml;
};
Generating go-deps.toml
tk sync regenerates it when go.mod or go.sum changes. To run the
generator yourself:
godeps-gen -o go-deps.toml
Options:
--no-prefetch: Skip fetching the Nix hashes of the modules' proxy.golang.org zips, the source the godeps cell fetches from (the hashes are then invalid)--no-cache: Always fetch from the network, bypassing the prefetch cache--indirect: Include indirect (transitive) dependencies (default: true)-o, --output: Output file (default: stdout)
Using Dependencies in Build Files
go_binary(
name = "hello",
srcs = ["main.go"],
deps = [
"godeps//vendor/github.com/spf13/cobra:cobra",
],
)
Multiple Modules and Local Replaces
Several Go modules in one repo are declared as a go.work workspace at the
project root. Go resolves them together into one go-deps.toml and one
godeps cell, and rules sync maps imports of a member to its targets in the
repo. A local-path replace directive must point at a workspace member.
See the Go language guide for detailed documentation.
External Fork Replace Directives
Turnkey also supports replace directives that point to external forks:
In go.mod:
replace github.com/original/pkg => github.com/myfork/pkg v1.2.3
In go-deps.toml (generated by godeps-gen):
[deps."github.com/original/pkg@v1.2.3"]
import_path = "github.com/original/pkg"
fetch_path = "github.com/myfork/pkg"
version = "v1.2.3"
hash = "sha256-..."
The cell builder fetches from fetch_path but stores under import_path, so
your code continues importing from the original path while using the fork's
source.
See the Go language guide for detailed documentation.
Rust Dependencies
Configuration
turnkey.toolchains.buck2.rust = {
enable = true;
depsFile = ./rust-deps.toml;
};
Generating rust-deps.toml
rustdeps-gen --cargo-lock Cargo.lock -o rust-deps.toml
Options:
--cargo-lock: Path to Cargo.lock file (default: Cargo.lock)--no-prefetch: Skip prefetching (produces incorrect hashes)--no-cache: Always fetch from the network, bypassing the prefetch cache-o, --output: Output file (default: stdout)
Handling Special Cases
Some Rust crates require additional configuration. See the Rust Dependency Handling guide for:
- Build scripts that emit rustc flags
- Generated source files
- Native code compilation
Python Dependencies
Configuration
turnkey.toolchains.buck2.python = {
enable = true;
depsFile = ./python-deps.toml;
};
Recommended Workflow (using uv)
# 1. Generate lock file from pyproject.toml
uv lock
# 2. Export to PEP 751 format
uv export --format pylock.toml -o pylock.toml
# 3. Generate python-deps.toml with Nix hashes
pydeps-gen --lock pylock.toml -o python-deps.toml
Input Formats
| Format | Flag | Reproducibility | Notes |
|---|---|---|---|
| pylock.toml (PEP 751) | --lock | Best | Exact versions and URLs |
| pyproject.toml | --pyproject | Varies | Uses latest matching versions |
| requirements.txt | --requirements | Varies | Pin versions with == for reproducibility |
CLI Options
--lock <PATH> Path to pylock.toml (PEP 751 lock file) - RECOMMENDED
--pyproject <PATH> Path to pyproject.toml
--requirements <PATH> Path to requirements.txt
-o, --output <PATH> Output file (default: stdout)
--no-prefetch Skip prefetching (produces placeholder hashes)
--no-cache Always fetch from the network, bypassing the prefetch cache
--include-dev Include dev dependencies from optional-dependencies.dev
Anti-Patterns to Avoid
Never Use vendorHash
Nix's buildGoModule has a vendorHash that hashes the output of
go mod vendor. This is problematic:
- Implementation-dependent: The hash changes based on which packages are actually imported
- Opaque: You can't know the hash without running the build and letting it fail
- Unstable: Adding a new import from an existing module can change the hash
Instead, use per-module fetching where each module has its own deterministic hash.
Never Vendor in Repository
Even temporarily. If you see a vendor/ directory in the repo, something is
wrong.
Never Compute Hashes from Vendored Output
The hash should come from the source (e.g., GitHub tarball), not from transformed/vendored output.
The Go, Rust, Python, Solidity and JavaScript Cells Are Real Directories
.turnkey/godeps, .turnkey/rustdeps, .turnkey/pydeps, .turnkey/soldeps and .turnkey/jsdeps are directories that tk materialize keeps in line with the cell index the shell builds (ADR 0004).
_store/<store path name>: one symlink per crate, Go module, Python distribution or Solidity package, to its own store path. It is only ever created or deleted, never pointed elsewhere.vendor/<crate>@<version>/rules.starandvendor/<crate>/rules.star:alias()targets that forward to a store link's crate. Labels such asrustdeps//vendor/anyhow:anyhoware unchanged.vendor/<name>/rules.star: one per Python distribution, forwarding to its store link (ADR 0010). Labels such aspydeps//vendor/six:sixare unchanged.vendor/<import path>/rules.star: one per Go package, forwarding to its package in its module's store link (ADR 0008). Labels such asgodeps//vendor/golang.org/x/sys/unix:unixare unchanged. Where a dependency imports ago.workmember's package, its alias forwards to the member's target in the repo instead.vendor/<name>@<version>/rules.star: one per npm package, forwarding to its files in its store link. The cell's rootrules.star, which the index carries, declares the package graph over them: an instance per pnpm snapshot, andjsdeps//:<npm name>per direct dependency (ADR 0012).vendor/<name>/rules.star: one per Solidity package, forwarding to its store link, beside links to the package's files, which nativeforgereads through the rootremappings.txt(ADR 0011). Labels are unchanged:soldeps//:<package>andsoldeps//:bundlecome from the cell's rootrules.star, which the index carries, andsoldeps//vendor/<name>:<package>from the alias package.
Only what a change touches is rewritten, so a dependency bump recompiles the bumped crate's, module's or distribution's dependents and re-runs only their tests. An npm package bump re-runs the instances that depend on it and their consumers. A Solidity package bump re-runs every Solidity action, since each one stages the whole bundle, and nothing in another language. Everything else stays cached, with no daemon restart, and plain buck2 reads the new version. Don't edit the directory: the shell rewrites it on every load.
- Switching over: the first shell load after upgrading turnkey replaces the old
.turnkey/<cell>symlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. - Going back to an older turnkey: run
rm -rf .turnkey/<cell>, then reload the shell. tk: warning: the rustdeps cell was built from another rust-deps.toml(orgodepsandgo-deps.toml,pydepsandpython-deps.toml,soldepsandsolidity-deps.toml,jsdepsandjs-deps.toml): the deps file changed since the shell last loaded. Rundirenv reload, or re-enter the shell, to rebuild the cell.
Troubleshooting
Dependencies Not Found
If Buck2 can't find a dependency:
-
Check that the deps TOML file is up to date:
tk sync -
Verify the cell exists:
ls -la .turnkey/godeps -
Check the target path format:
# Correct format godeps//vendor/github.com/spf13/cobra:cobra # Wrong - missing vendor/ prefix godeps//github.com/spf13/cobra:cobra
Hash Mismatch Errors
If you get hash mismatch errors when building:
-
Regenerate the deps file with fresh hashes:
godeps-gen --no-cache -o go-deps.toml -
Re-enter the dev shell:
exit nix develop
Stale Dependencies
If dependency changes aren't picked up:
-
Kill the Buck2 daemon, if you build with plain
buck2:buck2 killA running daemon keeps what it read through a repointed symlink (Symlinked Cells and Plain buck2);
tkrestarts it itself. -
Force a full sync:
tk sync
Dependency Fixups
Some dependencies don't build under Buck2 as they are. A Rust crate's
build.rs never runs, so whatever it generates, compiles or tells rustc has
to come from somewhere else; a dependency may need a patch. A fixup is
what turnkey supplies for one dependency to make it build: a patch, the
output its build script would generate, the flags it would pass.
Fixups come in fixup sets: modules of class turnkeyFixups
(ADR 0003).
Your repository brings the sets it needs, and its own fixups, through one
option. turnkey applies no fixups you didn't bring.
Bringing fixups
perSystem = { ... }: {
turnkey.toolchains.buck2.fixups = {
# Sets published by other flakes
imports = [
inputs.turnkey.modules.turnkeyFixups.serde
inputs.acme-fixups.modules.turnkeyFixups.default
];
# This repository's own fixups
rust.zerocopy.buildScript.skip = true;
};
};
A fixup is keyed by the dependency's name in its own ecosystem:
rust.<crate>, go."<import path>", python.<distribution>,
javascript.<package>, solidity.<package>.
turnkey's published fixups
turnkey publishes the fixups its own repository needs, one module per
family, under inputs.turnkey.modules.turnkeyFixups:
| Module | Crates |
|---|---|
serde | serde, serde_core, serde_json |
thiserror | thiserror |
ring | ring 0.17 |
rustix | rustix |
nix | nix |
fuser | fuser |
tree-sitter | tree-sitter and its rust, python, solidity, starlark and typescript grammars |
build-script-skips | crates whose build script turnkey's builds need nothing from |
default | all of the above |
Import only what your lock needs: an imported fixup for a crate you don't lock is silently unused.
Every build script needs a fixup
Buck2 never runs build.rs, so every crate you lock that has one needs a
fixup saying what stands in for it. The Rust cell fails to build otherwise:
error: turnkey: serde 1.0.228 has a build.rs, and no fixup says what stands in for it; a published fixup set does: add `inputs.turnkey.modules.turnkeyFixups.serde` to turnkey.toolchains.buck2.fixups.imports
When no published set accounts for the crate, the error says how to write the fixup. If the crate's build script only probes the compiler or the target, or emits cfgs for features you don't use, the build needs nothing from it:
rust.zerocopy.buildScript.skip = true;
Otherwise, give it a build script generating what its build.rs would, and
the flags it would pass. The developer manual's
Dependency Generators
page describes the whole record and how to diagnose what a crate needs.
Versions
A fixup applies to every locked version of its dependency. Fields that
hold only for some versions go in versions entries, whose bounds are
compared with the locked version:
rust.ring.versions = [
{
when = { atLeast = "0.17"; below = "0.18"; };
buildScript.generate = ...;
}
];
Every entry whose bounds hold applies. A version no entry covers gets only the fixup's other fields, so a new major version of a crate is reported as unaccounted for rather than built with a fixup written for another.
Patches
rust.some-crate.patches = [ ./patches/some-crate-fix.patch ];
go."github.com/foo/bar".patches = [ ./patches/bar.patch ];
A fixup's patches apply to the dependency's own source, in order, with
-p1: a plain git diff in a checkout of the dependency works as is.
Patches from several sets apply in import order, before a Rust crate's
build script runs.
Fixup patches are separate from the patches tk compose patch writes to
.turnkey/patches/<cell>/ from the FUSE edit layer. Those are this
repository's local, exact-version workarounds.
- Rust cell: each patch goes in its package's directory,
.turnkey/patches/rustdeps/vendor/<crate>@<version>/, and applies in that crate's own derivation, after its fixup. Changing a patch rebuilds only that crate and what depends on it.- A directory may also be named after the crate alone
(
vendor/anyhow/); it then goes to the version the cell's unversioned alias points at. - A patch that doesn't apply exactly, with no fuzz, fails the build and names the crate.
- A patch file left directly under
rustdeps/, from before this layout, fails evaluation: move it into its package's directory, or regenerate it withtk compose patch.
- A directory may also be named after the crate alone
(
- Go cell: each patch goes in its module's directory,
.turnkey/patches/godeps/vendor/<module path>/, and applies in that module's own derivation, after its fixup. - Python cell: each patch goes in its distribution's directory,
.turnkey/patches/pydeps/vendor/<name>/, and applies in that distribution's own derivation, after its fixup and before itsrules.staris written. Changing a patch rebuilds only that distribution and what depends on it.- Patches apply to the distribution's unpacked wheel, the installed
layout, not its sdist. A patch written against sdist paths (such as
vendor/requests/src/requests/...) doesn't apply: regenerate it withtk compose patch. - A patch that doesn't apply exactly, with no fuzz, fails the build and names the distribution.
- A patch file left directly under
pydeps/, from before this layout, fails evaluation: move it into its distribution's directory, or regenerate it withtk compose patch. So does a directory naming a distributionpython-deps.tomldoesn't hold.
- Patches apply to the distribution's unpacked wheel, the installed
layout, not its sdist. A patch written against sdist paths (such as
- Solidity cell: each patch goes in its package's directory,
.turnkey/patches/soldeps/vendor/<name>/(for a scoped npm package,vendor/@<scope>/<name>/, such asvendor/@openzeppelin/contracts/), and applies in that package's own derivation, after its fixup. Changing a patch rebuilds only that package.- A patch that doesn't apply exactly, with no fuzz, fails the build and names the package.
- A patch file left directly under
soldeps/, from before this layout, or in a directory that is no package's, fails evaluation: move it into its package's directory, or regenerate it withtk compose patch.
- JavaScript cell: each patch goes in its package's directory,
.turnkey/patches/jsdeps/vendor/<name>@<version>/(for a scoped package,vendor/@<scope>/<name>@<version>/), and applies in that package's own derivation, after its fixup, so to every instance of it. Changing a patch rebuilds only that package.- A directory may also be named after a direct dependency alone
(
vendor/lodash/); it then goes to the version the rootpackage.jsonresolves it to. - A patch that doesn't apply exactly, with no fuzz, fails the build and names the package.
- A patch file left directly under
jsdeps/, from before this layout, or in a directory that is no package's, fails evaluation: move it into its package's directory, or regenerate it withtk compose patch.
- A directory may also be named after a direct dependency alone
(
When sets disagree
Fixups merge field by field, as NixOS modules do: flags and patches from every set concatenate. Two sets that give a crate different build scripts, or an environment variable different values, fail evaluation, naming both files:
error: The option `rust.serde.buildScript.generate.<function body>' has conflicting definition values:
- In `conflicting/flake.nix#modules.turnkeyFixups.default': "echo another serde"
- In `acme/flake.nix#modules.turnkeyFixups.default': "..."
Resolve it in your own fixups:
- Override one field with
lib.mkForce:rust.serde.buildScript.generate = lib.mkForce "..."; - Drop one fixup an imported set brings:
rust.ring.enable = false; - Drop a whole module:
disabledModules = [ ... ];, or don't import it.
Unused fixups
A fixup written in your own turnkey.toolchains.buck2.fixups that matches
no locked dependency, or a versions entry matching none of its locked
versions, warns at evaluation: it usually means a rename or an upgrade left
it behind. Fixups from imported sets never warn, so one organization-wide
set can serve many repositories that each lock only part of it.
Publishing a fixup set
A fixup set is a plain module; publish it from any flake as
modules.turnkeyFixups.<name>. With flake-parts, import its modules
module, which stamps the module's class:
{
imports = [ inputs.flake-parts.flakeModules.modules ];
flake.modules.turnkeyFixups = {
openssl = ./fixups/openssl.nix;
default = { imports = [ ./fixups/openssl.nix ]; };
};
}
Without flake-parts, set the class yourself:
outputs = { self, ... }: {
modules.turnkeyFixups.default = {
_class = "turnkeyFixups";
imports = [ ./fixups/openssl.nix ];
};
};
A set's module receives pkgs and lib, and may hold fixups for several
languages at once, such as for a library packaged for both Rust and
Python:
# fixups/acme-proto.nix
{ ... }:
{
rust.acme-proto = {
buildScript.skip = true;
patches = [ ./acme-proto-rust.patch ];
};
python.acme-proto.patches = [ ./acme-proto-python.patch ];
}
Python Workspaces
Turnkey lays out Python source as a uv workspace, in parallel to the Cargo workspace pattern used for Rust. A single uv.lock resolves every Python package in the monorepo against a consistent dependency set, while each package keeps its own pyproject.toml declaring exactly what it consumes.
Two tracks run side by side over the same source:
- uv track —
uv sync,uv run, IDE language servers, REPL. Members are installed editable so source edits are reflected immediately. - Buck2 track —
tk build,tk test. External packages are vendored into thepydepscell built frompython-deps.toml.
Repository Layout
/repo/
├── pyproject.toml # Workspace root: members + uv.lock anchor
├── uv.lock # Single resolved lockfile (managed by uv)
├── pylock.toml # PEP 751 export from uv.lock
├── python-deps.toml # Generated for Buck2/Nix from pylock.toml
└── src/python/<member>/
├── pyproject.toml # [project] + hatchling build backend
├── rules.star # Buck2 targets for the member
└── turnkey/<member>/ # Source under shared turnkey.* namespace
├── __init__.py
└── ...
Tests live in a sibling tests/ directory inside each member, kept outside the importable namespace.
The turnkey.* Namespace Convention
Every workspace member contributes a subpackage under the shared turnkey PEP 420 implicit namespace package. No member defines a top-level turnkey/__init__.py; Python's import system resolves turnkey.parser, turnkey.config, etc. by walking every sys.path entry that exposes a turnkey/<name>/ directory.
Downstream Projects: Pick Your Own Namespace
The turnkey.* prefix is this repository's namespace. If you adopt the same workspace pattern in a different monorepo, choose a namespace specific to your organisation — e.g. acme.<name> — to avoid colliding with packages on PyPI or other turnkey-based repos. The mechanics are identical; substitute turnkey for your namespace throughout this guide.
Member pyproject.toml
Each library member uses the hatchling backend and points it at the turnkey/ directory:
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "turnkey-parser"
version = "0.1.0"
description = "Cargo manifest and feature-graph utilities"
requires-python = ">=3.11"
dependencies = [
"turnkey-config", # cross-member dep
]
[tool.uv.sources]
turnkey-config = { workspace = true }
[tool.hatch.build.targets.wheel]
packages = ["turnkey"] # everything under turnkey/<name>/ is the wheel content
packages = ["turnkey"] is the key line: it tells hatchling that the wheel's content is whatever lives under the turnkey/ directory of this member. Combined with PEP 420 namespace resolution, every member ships only its own turnkey/<name>/ slice without anyone owning turnkey/__init__.py.
Cross-Member Dependencies
Declare the dep under [project] dependencies with the bare package name, then pin its source to the workspace under [tool.uv.sources]:
dependencies = ["turnkey-config"]
[tool.uv.sources]
turnkey-config = { workspace = true }
This mirrors Cargo.toml's serde.workspace = true pattern — the consumer member doesn't pin a version, the lockfile reconciles it.
External Dependencies
Declare externals in the member that consumes them, never the workspace root:
# src/examples/python-hello-deps/pyproject.toml
[project]
name = "turnkey-example-python-hello-deps"
dependencies = ["six>=1.16.0"]
The single uv.lock at the workspace root resolves every external version-consistently across members.
Non-Packaged Members
Some members exist only to declare dependencies, not to be installed (typical for application-like entrypoints or examples). Mark them non-packaged:
[project]
name = "turnkey-example-python-hello-deps"
version = "0.1.0"
dependencies = ["six>=1.16.0"]
[tool.uv]
package = false # uv won't build/install this member
No [build-system] is required. uv still resolves the member's dependencies as part of the workspace lock.
Root pyproject.toml
The workspace root anchors membership and the shared lockfile:
[project]
name = "turnkey"
version = "0.1.0"
requires-python = ">=3.11"
# Listing members as dependencies makes the default `uv sync` install all
# of them in one shot — no `--all-packages` flag needed.
dependencies = [
"turnkey-parser",
"turnkey-config",
"turnkey-example-python-hello",
"turnkey-example-python-hello-deps",
]
[dependency-groups]
# Dev tooling — auto-installed by 'uv sync' so 'uv run pytest' Just Works.
dev = ["pytest>=7.0"]
[tool.uv.workspace]
members = [
"src/python/parser",
"src/python/config",
"src/examples/python-hello",
"src/examples/python-hello-deps",
]
[tool.uv.sources]
turnkey-parser = { workspace = true }
turnkey-config = { workspace = true }
turnkey-example-python-hello = { workspace = true }
turnkey-example-python-hello-deps = { workspace = true }
[tool.uv]
package = false # the root itself isn't a packaged project
Buck2 Integration
Member source paths are spelled relative to the member's rules.star:
load("@prelude//:rules.bzl", "python_library", "python_test")
python_library(
name = "parser",
srcs = [
"turnkey/parser/__init__.py",
"turnkey/parser/grammar.py",
"turnkey/parser/tokens.py",
],
base_module = "",
deps = ["//src/python/config:config"],
visibility = ["PUBLIC"],
)
python_test(
name = "test_parser",
srcs = ["tests/test_parser.py"],
base_module = "tests",
deps = [":parser"],
)
base_module = "" tells Buck2 to install sources at their declared srcs paths, so files land at turnkey/parser/... in the runtime tree — matching the import prefix the rest of the codebase uses.
Adding or Updating Dependencies
# 1. Edit the member that needs the dep
$EDITOR src/python/parser/pyproject.toml # add to [project] dependencies
# 2. Refresh editable installs (optional but recommended)
uv sync
# 3. Refresh the Buck2 pipeline
tk sync
uv add and uv remove do the same in one step: the uv wrapper runs
tk sync itself when they change uv.lock.
tk sync runs two rules, in order:
- pylock re-exports
pylock.tomlfromuv.lockwhenuv.lockorpyproject.tomlis newer, relocking first ifpyproject.tomlchanged:uv export --all-packages --no-dev --format pylock.toml.--all-packagesincludes externals from every member, and--no-devkeeps dev tooling (pytest etc.) out of the pydeps cell. - python regenerates
python-deps.tomlfrompylock.toml, and fromuv.lockthe dependency graph: each dependency's environment marker and each package's extras.
The pylock rule exists when the flake sets buck2.python.uvLockFile
(turnkey's own flake sets it to uv.lock) along with
buck2.python.lockFile.
One Version per Distribution
The pydeps cell holds one version of each distribution, at
pydeps//vendor/<name>:<name>
(ADR 0010).
uv can lock several: when the resolution forks on a marker, for example
numpy 1.x for python_version < '3.10' and 2.x above, the lock holds one
entry per fork. pydeps-gen then fails, writes nothing, and names the
distribution with each locked version and its marker:
pylock.toml locks several versions of one distribution, and the pydeps cell holds one version per distribution.
2 versions of numpy:
numpy 1.26.4 (python_full_version < '3.10')
numpy 2.1.0 (python_full_version >= '3.10')
Pin it, in the pyproject.toml that depends on it, to a range one version satisfies on every Python the workspace allows, so the lock no longer forks; then run tk sync.
Pin the dependency so the lock no longer forks: constrain it, in the
pyproject.toml of the member that depends on it, to a range one version
satisfies for every Python the workspace allows (numpy>=2.1), or raise the
workspace's requires-python so the fork's other branch can't happen. Then
run tk sync, which relocks and regenerates python-deps.toml.
Running Code
| Task | uv track | Buck2 track |
|---|---|---|
| Run all tests | uv run pytest | tk test //src/python/... |
| Run a single member's tests | uv run pytest src/python/parser | tk test //src/python/parser:test_parser |
| Run an example | uv run --package <pkg-name> <script> | tk run //src/examples/python-hello-deps:python-hello-deps |
| REPL with members available | uv run python | n/a |
| IDE language server | Point at .venv/bin/python | n/a |
Both tracks resolve external dependencies the same way (uv.lock is the single source of truth), but the install paths differ: the uv track installs into .venv/, the Buck2 track materialises external packages into .turnkey/pydeps/vendor/<name>/.
See Also
- Managing Dependencies — overall dependency flow across all languages.
- Python — Buck2 build rules for Python targets.
Go Support
Turnkey provides comprehensive Go support with Buck2 integration.
Setup
Add to toolchain.toml:
[toolchains]
go = {}
godeps-gen = {}
Enable Go dependencies in flake.nix:
turnkey.toolchains.buck2.go = {
enable = true;
depsFile = ./go-deps.toml;
};
Project Structure
my-project/
├── go.mod
├── go.sum
├── go-deps.toml # Generated from go.mod
├── cmd/
│ └── myapp/
│ ├── main.go
│ └── rules.star
└── pkg/
└── mylib/
├── lib.go
└── rules.star
Build Rules
In rules.star:
load("@prelude//go:go.bzl", "go_binary", "go_library")
go_binary(
name = "myapp",
srcs = ["main.go"],
deps = ["//pkg/mylib:mylib"],
)
Tests
A go_test compiles its srcs with the package of its target_under_test,
as go test does:
go_test(
name = "mylib_test",
srcs = glob(["*_test.go"]),
target_under_test = ":mylib",
)
Test files can be in the package itself (package mylib) or in an external
test package (package mylib_test), which sees only the package's exported
API. Both kinds can be in the same go_test.
External Dependencies
Reference third-party packages via the godeps cell:
go_library(
name = "mylib",
srcs = ["lib.go"],
deps = ["godeps//vendor/github.com/pkg/errors:errors"],
)
Auto-Sync
The go command is wrapped to auto-sync dependencies:
go get github.com/some/package # Triggers sync
go mod tidy # Triggers sync
Multiple Modules: go.work
A repo with several Go modules declares them as a go.work workspace at the project root. This is the only supported multi-module shape (ADR 0007). Go resolves all the members together, so each third-party module has one version across the repo and one godeps cell. A repo without a go.work is a workspace of one member: the root go.mod.
my-monorepo/
├── go.work # use ./app ./shared-lib ./third_party/cobra-fork
├── go-deps.toml # Generated by tk sync, lists the members
├── app/
│ ├── go.mod # module github.com/company/app
│ └── cmd/server/
│ ├── main.go # imports github.com/company/shared-lib/log
│ └── rules.star
├── shared-lib/
│ ├── go.mod # module github.com/company/shared-lib
│ └── log/
│ ├── log.go
│ └── rules.star
└── third_party/cobra-fork/ # a patched local fork of github.com/spf13/cobra
├── go.mod # module github.com/spf13/cobra
└── rules.star
How imports map
The members are the only first-party Go code. tk sync records them in go-deps.toml under [members], and rules sync maps each import to the member whose module path is its longest prefix on a / boundary:
| Import | Target |
|---|---|
github.com/company/shared-lib/log | //shared-lib/log:log |
github.com/spf13/cobra | //third_party/cobra-fork:cobra |
github.com/spf13/cobra/doc | //third_party/cobra-fork/doc:doc |
github.com/company/shared-libx | not a member: godeps//vendor/... |
A package's target is named after its import path's last component, as in the godeps cell. That matters at a member's root: the root package of github.com/spf13/cobra in third_party/cobra-fork is :cobra, not :cobra-fork after its directory.
Local forks and replace
A member can be a local fork of a third-party module: give it the upstream module path and add it to go.work. Go then builds every import of that module from the fork, including imports from other third-party modules. In the godeps cell, those imports reach the member's targets through forwarding alias packages (ADR 0008).
A local-path replace directive, in go.work or a member's go.mod, must point at a workspace member. tk sync fails on one that points anywhere else: add the directory to go.work.
Modules outside the workspace
A go.mod that isn't a member is outside the Go build, as it is for go build ./... in the workspace. Rules sync leaves the Go rules under it alone. Test fixtures with their own go.mod need no fencing.
External Fork Replacements
Turnkey supports replace directives that point to external forks (not local paths). This is useful when:
- Using a forked version of a dependency with bug fixes
- Using a maintained fork of an abandoned project
- Testing changes before upstreaming
How It Works
When godeps-gen encounters an external replace directive like:
replace github.com/original/pkg => github.com/myfork/pkg v1.2.3
It will:
- Set the dependency's
import_pathtogithub.com/original/pkg(for correct imports) - Set the
fetch_pathtogithub.com/myfork/pkg(where to actually fetch from) - Use the replacement version
In go.mod
module github.com/company/myapp
require github.com/original/pkg v1.0.0
replace github.com/original/pkg => github.com/myfork/pkg v1.2.3
Generated go-deps.toml
[deps."github.com/original/pkg@v1.2.3"]
import_path = "github.com/original/pkg"
fetch_path = "github.com/myfork/pkg"
version = "v1.2.3"
hash = "sha256-..."
How the Cell Builder Uses This
The Nix cell builder:
- Fetches the source from
fetch_path(the fork) - Stores it in the vendor directory under
import_path(the original path) - Generates Buck2 rules using the original import path
This means your code continues to import from the original path (github.com/original/pkg), but the actual source comes from your fork.
Version Handling
External replaces can change the version:
| go.mod replace | Result |
|---|---|
=> github.com/fork v1.2.3 | Uses v1.2.3 from fork |
=> github.com/fork (no version) | Uses the required version from fork |
Version-specific replaces are also supported:
// Only replace v1.0.0, not other versions
replace github.com/pkg v1.0.0 => github.com/fork/pkg v1.0.1
Common Use Cases
Using a fork with a fix:
// Your fork has a critical bug fix not yet merged upstream
replace github.com/upstream/logger => github.com/you/logger v1.0.1-patched
Using a maintained fork:
// Original project abandoned, using community fork
replace github.com/old/abandoned => github.com/community/maintained v2.0.0
Testing before upstreaming:
// Test your changes before creating a PR
replace github.com/original/pkg => github.com/you/pkg v0.0.0-20240101
Rust Support
Turnkey provides Rust support with automatic dependency management.
Setup
Add to toolchain.toml:
[toolchains]
rust = {}
cargo = {}
rustdeps-gen = {}
Enable Rust dependencies in flake.nix:
turnkey.toolchains.buck2.rust = {
enable = true;
depsFile = ./rust-deps.toml;
};
Project Structure
my-project/
├── Cargo.toml
├── Cargo.lock
├── rust-deps.toml # Generated by tk sync, from cargo
└── rust/
└── mycrate/
├── src/
│ └── lib.rs
└── rules.star
Build Rules
In rules.star:
load("@prelude//rust:rust.bzl", "rust_library", "rust_binary")
rust_library(
name = "mycrate",
srcs = glob(["src/**/*.rs"]),
deps = ["rustdeps//serde:serde"],
)
External Dependencies
Reference crates via the rustdeps cell:
deps = [
"rustdeps//serde:serde",
"rustdeps//tokio:tokio",
]
Features and Dependencies
Each vendored crate is built with the features and dependencies cargo
resolves for it. tk sync records, for every crate, its package slice in
rust-deps.toml: the features cargo tree reports for cargo test --workspace, and its normal dependencies resolved to exact versions, each
with the platforms it applies on (one cargo tree run per platform the
project builds for). Each crate's own rules.star is generated from its slice
alone, so a change to one crate rebuilds only that crate and its dependents.
Declare what you need in Cargo.toml as you would for Cargo; there is
nothing to repeat for Buck2, and no way to override Cargo's result: ask for a
feature in the member's Cargo.toml.
tk syncrunscargowith--locked, soCargo.lockmust matchCargo.toml. After editing a manifest by hand, run a cargo command (ortw cargo …) that updates the lock. The first run downloads the crates into~/.cargo/registry.- Every workspace member's
Cargo.tomlis a source ofrust-deps.toml, so a features-only edit in a member regenerates it. - A crate no configured platform builds (Windows-only, wasm, a build
dependency) has an empty slice: its
rules.starhas no features or dependencies, and nothing configures it.
The rustdeps cell is a real directory, .turnkey/rustdeps, kept in line
with the Nix-built cell index by tk materialize; see
Managing Dependencies.
Auto-Sync
The cargo command is wrapped to auto-sync:
cargo add serde # Triggers sync
Python Support
Turnkey provides Python support with Buck2 integration.
Python source in this repo is laid out as a uv workspace, with each package owning its own
pyproject.tomland contributing to a sharedturnkey.*PEP 420 namespace. This page covers the Buck2 build rules; read the workspace workflow guide first for the overall layout and the uv/Buck2 dual-track model.
Setup
Add to toolchain.toml:
[toolchains]
python = {}
uv = {}
pydeps-gen = {}
Enable Python dependencies in flake.nix:
turnkey.toolchains.buck2.python = {
enable = true;
depsFile = ./python-deps.toml;
};
Project Structure
my-project/
├── pyproject.toml
├── uv.lock
├── python-deps.toml # Generated from uv.lock
└── python/
└── mypackage/
├── __init__.py
├── main.py
└── rules.star
Build Rules
In rules.star:
load("@prelude//python:python.bzl", "python_library", "python_binary", "python_test")
python_library(
name = "mypackage",
srcs = glob(["**/*.py"]),
deps = ["pydeps//requests:requests"],
)
python_binary(
name = "main",
main = "main.py",
deps = [":mypackage"],
)
python_test(
name = "test",
srcs = ["test_main.py"],
deps = [":mypackage"],
)
External Dependencies
Reference packages via the pydeps cell:
deps = [
"pydeps//requests:requests",
"pydeps//click:click",
]
python-deps.toml
pydeps-gen writes it from pylock.toml (what to fetch) and uv.lock (the
dependency graph). Schema 3 records each distribution's pure wheel, with
markers and extras:
schema_version = 3
[deps.requests]
version = "2.32.3"
# The Nix hash of the unpacked wheel
hash = "sha256-..."
# The locked py3-none-any wheel
url = "https://files.pythonhosted.org/.../requests-2.32.3-py3-none-any.whl"
# The lock's marker for installing the package at all, when it has one
marker = "python_version >= '3.8'"
# Its dependencies: each package's key, with its marker and the extras it
# asks for, when it has them
dependencies = [
{ name = "urllib3" },
{ name = "colorama", marker = "sys_platform == 'win32'" },
]
# The extras some package or workspace member asks it for
requested_extras = ["socks"]
# Its extras, and the dependencies each adds
[deps.requests.extras]
"socks" = [{ name = "pysocks" }]
The pydeps cell uses them: a package's target depends on its dependencies
and on those of its requested extras, each where its marker holds. Markers
are evaluated on every platform in buck2.platforms, for the Python
toolchain's version: a dependency some platforms get is a select() on the
platform, and one none gets is left out.
A file from before schema 3 records sdists, and fails evaluation asking for
tk sync, which regenerates it.
Distributions are their locked wheels
The pydeps cell vendors each distribution as the pure (py3-none-any)
wheel uv locked for it, unpacked: the layout an installer puts in
site-packages
(ADR 0013).
So import requests works although requests keeps its code under src/ in
its sdist, and an sdist's setup.py, tests/ and docs are not in the
target.
- The wheel's
<name>.data/purelibandplatlibare merged into its root, as an installer would; the rest of<name>.data/(scripts, headers, data) is dropped. Fixups, then user patches, apply to that tree. - The distribution's
python_libraryholds everything the wheel installs: its.pyfiles assrcs, and every other file asresources. That includes data files (such ascertifi'scacert.pem) and the*.dist-infodirectory, soimportlib.metadata.version("requests")and entry points work. - A distribution with no pure wheel fails
pydeps-gen, which names it: one with only platform-specific (compiled) wheels, and one with only an sdist. turnkey doesn't build sdists into wheels, and doesn't vendor platform wheels yet (#248).
src/examples/python-requests uses requests and certifi this way.
Markers and Extras
Rules sync reads a workspace member's pyproject.toml as well as its
imports:
- A dependency in
[project] dependencieswith a platform marker (sys_platform,platform_system,platform_machine,os_name) is written as aselect()over the platforms, like any platform-conditional dep. - Other markers (
python_version,implementation_name, ...) are fixed by the Python toolchain: they are evaluated once, for thepython3of the shell, and a dependency whose marker doesn't hold is left out. - Extras are a variant: a target declares the extras it's built with on
turnkey's prelude attribute
extras(possibly aselect()), and sync adds the dependencies they enable, whether its sources import them or not. A dependency declared only as an extra's is kept only on a target built with that extra.
python_library(
name = "app-full",
extras = ["socks"],
deps = [...], # sync adds the socks extra's dependencies
)
Auto-Sync
The uv command is wrapped to auto-sync:
uv add requests # Triggers sync
TypeScript Support
Turnkey provides TypeScript support via custom Buck2 rules.
Setup
Add to toolchain.toml:
[toolchains]
nodejs = {}
typescript = {}
Project Structure
my-project/
└── ts/
└── myapp/
├── src/
│ └── index.ts
├── tsconfig.json # Optional
└── rules.star
Build Rules
In rules.star:
load("@prelude//typescript:typescript.bzl", "typescript_binary", "typescript_library")
typescript_library(
name = "lib",
srcs = glob(["src/**/*.ts"]),
)
typescript_binary(
name = "myapp",
main = "src/index.ts",
srcs = glob(["src/**/*.ts"]),
deps = [":lib"],
)
Running TypeScript
tk run //ts/myapp:myapp
A typescript_test takes the same attributes as a typescript_binary. It
compiles the code and runs it with node: the test passes when it exits 0.
typescript_test(
name = "myapp_test",
main = "src/index.test.ts",
srcs = glob(["src/**/*.ts"]),
)
tk test //ts/myapp:myapp_test
Configuration
The TypeScript toolchain uses sensible defaults. For custom configuration, provide a tsconfig.json:
typescript_binary(
name = "myapp",
main = "src/index.ts",
srcs = glob(["src/**/*.ts"]),
tsconfig = "tsconfig.json",
)
npm Dependencies
npm packages are declared in the root package.json and locked in
pnpm-lock.yaml. tk sync runs jsdeps-gen, which writes
js-deps.toml, and the jsdeps cell is built from it
(ADR 0012).
Rules sync writes a target's npm_deps from the packages its sources
import.
Labels
A target lists the packages it imports in its npm_deps, by their npm
name, verbatim:
typescript_binary(
name = "app",
main = "main.ts",
srcs = ["main.ts"],
npm_deps = [
"jsdeps//:lodash",
"jsdeps//:@types/lodash",
],
)
Only the root package.json's dependencies (js-deps.toml's [direct])
have such a label, at the version the root resolves them to. Whatever they
depend on is in the cell too, but code can't import it, just as pnpm keeps
it out of the root node_modules: declare a package in package.json to
import it. @types/... packages are usually devDependencies: set
buck2.javascript.includeDevDependencies = true so they are direct too.
Labels used to name a scoped package with the @ dropped and / as _
(jsdeps//:types_lodash). Rules sync replaces them with the npm names
(Upgrading).
The node_modules each target sees
Each locked name@version is fetched once, as its own store path. Each
instance of it, one per pnpm snapshot, is a target in the cell: a package
that pnpm installs once per peer resolution (react-dom with React 17 and
with React 18) has one instance per resolution, and two versions of one
name are two instances.
An instance's output is a real node_modules/<name>/ directory, copied
from the package's files, with each of its dependencies a relative symlink
beside it, into that dependency's instance. A target's node_modules links
each of its npm_deps to its instance's package directory. node and tsc
find a package's own dependencies by walking node_modules up from its
real path, as in pnpm's node_modules/.pnpm, so each package sees exactly
the dependencies its lock entry resolves, with no --preserve-symlinks and
no NODE_PATH. A typescript_binary's output has its own node_modules
link beside the compiled code, so node dist/main.js resolves the same
way, for CommonJS (.cts, compiled to .cjs) and ES modules (.mts, to
.mjs) alike.
Instances that depend on each other in a cycle are laid out together, as one target with a forwarding target per instance.
A bump of one package rebuilds its store path and re-runs only the
instances that depend on it, and their consumers: .turnkey/jsdeps is a
real directory that tk materialize keeps in line with the cell index, so
plain buck2 reads the new version without a daemon restart.
Copying costs disk: every installed package is copied once per
configuration under buck-out.
js-deps.toml
One [[package]] per locked name@version (its contents), one
[[instance]] per pnpm snapshot (its dependencies, by the name it imports
each as, resolved to instance keys), and [direct]:
[[package]]
name = "chokidar"
version = "3.6.0"
url = "https://registry.npmjs.org/chokidar/-/chokidar-3.6.0.tgz"
integrity = "sha512-..."
[[package]]
name = "fsevents"
version = "2.3.3"
url = "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz"
integrity = "sha512-..."
os = ["darwin"]
[[instance]]
key = "chokidar@3.6.0"
name = "chokidar"
version = "3.6.0"
[instance.dependencies]
anymatch = "anymatch@3.1.3"
braces = "braces@3.0.3"
[instance.optional_dependencies]
fsevents = "fsevents@2.3.3"
[direct]
chokidar = "chokidar@3.6.0"
- An instance's
keyis pnpm's snapshot key verbatim, peer groups included (react-dom@18.2.0(react@18.2.0)). Its target in the cell is named as pnpm names its directory innode_modules/.pnpm(react-dom@18.2.0_react@18.2.0). os,cpuandlibcare the package's own restrictions, as npm names them (darwin,x64,glibc, or!win32to exclude one). A field that's absent allows anything.link:andfile:dependencies failjsdeps-gen, naming the package.
Packages from a registry Nix can't reach
A package locked from a registry that Nix can't fetch from, such as a
local one, can be kept in the project as its tarball, keyed by the URL
pnpm-lock.yaml records for it. The cell reads the file instead of
fetching, and still checks it against the lock's integrity:
buck2.javascript.tarballs = {
"http://localhost:4873/-/acme-utils-1.0.0.tgz" = ./registry/acme-utils-1.0.0.tgz;
};
Platform-specific packages
An instance whose optional dependencies install on some platforms only,
such as esbuild's per-platform binaries or chokidar's fsevents, is
resolved in the cell, for every platform in buck2.platforms: the
optional dependencies the platform gets (their os and cpu allow it,
and on Linux their libc allows glibc) are linked beside it, as a
select() keyed like rules sync's, with no DEFAULT. A consumer lists the
package unconditionally. src/examples/typescript-platform-deps is an
example.
Patching a package
User patches go under .turnkey/patches/jsdeps/vendor/<name>@<version>/,
or vendor/<name>/ for the version a direct dependency resolves to, and
apply to that package's files with --fuzz=0
(Fixups).
Solidity Support
Turnkey provides Solidity smart contract support with Buck2 integration, including compilation, testing with Foundry, and dependency management.
Setup
Add to toolchain.toml:
[toolchains]
solidity = {}
foundry = {}
Enable Solidity dependencies in flake.nix (if using external libraries):
turnkey.toolchains.buck2.solidity = {
enable = true;
depsFile = ./solidity-deps.toml;
};
Project Structure
my-project/
├── foundry.toml # The only Foundry configuration, for the whole repository
├── solidity-deps.toml # Generated dependency manifest
├── remappings.txt # Generated from solidity-deps.toml
└── src/
└── contracts/
├── rules.star
├── src/
│ └── MyToken.sol
└── test/
└── MyToken.t.sol
The repository has at most one foundry.toml, at its root. A package under
src/ has none of its own: a nested foundry.toml would become forge's root for its
subtree and break native forge there. With the tk.foundryConfigCheck option
on, a pre-commit hook rejects one (see Native forge).
Build Rules
forge drives both solidity_library and solidity_test, and reads the same
configuration native forge reads. The root foundry.toml holds every compiler
and test setting (optimizer, optimizer_runs, evm_version, via_ir,
[fuzz] runs, ...), and the root remappings.txt every remapping (overrides
go in foundry.toml, see Remappings). The rules take neither
settings nor remappings as attributes. So tests run against the bytecode that
ships, and native forge build produces the same bytecode as Buck2.
Each Solidity action runs forge in a scratch project that holds only its declared inputs, each at its place in the repository:
- the root
foundry.tomlandremappings.txt; - the target's sources, and those of every
solidity_libraryit depends on; - the
soldepscell'sbundle(every vendor package), at the cell link's path,.turnkey/soldeps/, where the rootremappings.txtpoints.
forge runs with the toolchain's solc (--use), --offline, and without the
caller's FOUNDRY_*/DAPP_* variables or ~/.foundry configuration.
The solidity_library and solidity_test macros add these inputs
themselves, from the [solidity] section turnkey writes into the generated
.buckconfig: foundry.toml always, the bundle and remappings.txt when
the repository has a soldeps cell. foundry.toml must be at the repository
root (turnkey.toolchains.buck2.solidity.foundryTomlFile's default); the
shell fails to evaluate otherwise.
Breaking change: a repository using the Solidity rules must export both files from a
rules.starat its root:# rules.star, at the repository root export_file(name = "foundry.toml", visibility = ["PUBLIC"]) export_file(name = "remappings.txt", visibility = ["PUBLIC"])Without them, a Solidity target fails to build with an unknown
root//:foundry.tomltarget. The rules also no longer acceptoptimizer,optimizer_runs,fuzz_runs,solc_versionorremappings: set them in the rootfoundry.toml.
Changing a setting in foundry.toml, or a dependency, rebuilds and retests
every Solidity target.
solidity_library
Compile Solidity source files with forge build <srcs>:
load("@prelude//solidity:solidity.bzl", "solidity_library")
solidity_library(
name = "my_token",
srcs = ["src/MyToken.sol"],
deps = ["soldeps//:openzeppelin_contracts"],
)
Its output is forge's artifacts directory (out/): one
<File>.sol/<Contract>.json per contract compiled, imports included.
solidity_contract
Extract a specific contract from a compiled library:
load("@prelude//solidity:solidity.bzl", "solidity_contract")
solidity_contract(
name = "my_token_artifact",
contract = "MyToken", # Contract name in source
lib = ":my_token",
)
It takes the artifact of the contract of that name defined in one of the library's own sources (not in an import); if two of them define it, split the library. It produces what solc's own outputs would be:
{contract}.abi- Contract ABI (JSON){contract}.bin- Deployment bytecode (hex){contract}.bin-runtime- Runtime bytecode (hex){contract}.metadata.json- Compiler metadata
solidity_test
Run tests with Foundry's forge test:
load("@prelude//solidity:solidity.bzl", "solidity_test")
solidity_test(
name = "my_token_test",
srcs = ["test/MyToken.t.sol"],
deps = [":my_token"],
)
Fuzz runs come from foundry.toml's [fuzz] runs. The fuzz seed is fixed per
target, unless the target opts out of test result caching (see
Test result caching).
External Dependencies
tk sync collects Solidity dependencies from two places, the root
foundry.toml and the root package.json, into the generated
solidity-deps.toml, and generates the root remappings.txt from that. Both
files are committed. Only direct dependencies count: tk sync does not follow a
dependency's own [dependencies], submodules or remappings.txt, so anything a
dependency needs is declared at the root too.
Git dependencies
Declare git dependencies in the root foundry.toml's [dependencies], in
Soldeer's table form:
[dependencies]
forge-std = { version = "1.8.0", git = "https://github.com/foundry-rs/forge-std", tag = "v1.8.0" }
solady = { version = "0.1.26", git = "https://github.com/vectorized/solady", rev = "<40-hex commit>" }
versionis required. It is the package's version insolidity-deps.toml, and what fixup sets match on.gitis the repository URL.- The pin is exactly one of
tag,branchorrev. Arevmust be a full 40-character commit hash:tk syncresolves pins withgit ls-remote, which lists refs but cannot expand an abbreviated commit without a clone. Pin a name withtagorbranchinstead.
Turnkey never runs Soldeer; soldeps-gen reads the table itself. A string
value, such as a Soldeer registry version (forge-std = "1.9.7") or the older
"<url>@<ref>" form, is rejected with an error showing the table form. So is
Soldeer's url = "…" form, since URL dependencies are not supported, and any
other unknown key, such as a misspelt pin.
npm packages
Solidity packages published to npm, such as @openzeppelin/contracts, are
ordinary dependencies in the root package.json, at the versions and integrity
hashes pnpm-lock.yaml pins. A package.json dependency counts as a Solidity
dependency when its tarball contains .sol files; there is no list of known
packages. To tell, tk sync downloads the tarball, checks it against the lock's
integrity (and fails on a mismatch), and looks for .sol files, within
generous size limits. solidity-deps.toml remembers the verdict for each lock
integrity, or for each name and version when the lock records no integrity:
a Solidity package is recorded as a [[package]], any other dependency as a
[[not_solidity]] entry. A recorded verdict is always reused, so a sync only
downloads the tarballs of packages that are new or that changed. When a
download fails, or --no-prefetch rules it out, the sync fails instead of
guessing.
Using a dependency
Reference a package through the soldeps cell:
solidity_library(
name = "my_token",
srcs = ["MyToken.sol"],
deps = ["soldeps//:openzeppelin_contracts"],
)
and import it under its own name:
import "@openzeppelin/contracts/token/ERC20/ERC20.sol";
import "forge-std/Test.sol";
Syncing
tk sync regenerates solidity-deps.toml whenever foundry.toml,
package.json or pnpm-lock.yaml changes (the paths are the
turnkey.toolchains.buck2.solidity options foundryTomlFile, packageJsonFile
and pnpmLockFile), then the root remappings.txt, next to foundry.toml,
from solidity-deps.toml. It runs soldeps-gen, which prefetches: it pins each
git dependency to the commit its tag, branch or rev resolves to. For a GitHub
repository it also records that commit's source archive and its Nix hash, so the
soldeps cell fetches it as a fixed-output derivation. An npm package that
pnpm-lock.yaml gives no integrity for gets the hash of its tarball. Hashes go
through turnkey's prefetch cache, so a regeneration only fetches what changed.
Remappings
Each package is imported under its own name, <name>/, with no aliases:
forge-std/... resolves inside forge-std. An npm package maps to its root. A
git dependency maps to the directory its own foundry.toml names as
[profile.default] src, to forge's default src/ when its foundry.toml sets
none, and to the repository root when it has no foundry.toml. soldeps-gen
reads that file at the pinned commit while prefetching, and records the result
as the package's remapping in solidity-deps.toml. Later syncs reuse the
recorded target while the package's pin is unchanged, so a sync with
--no-prefetch works for them; a new or re-pinned git dependency needs a
prefetching sync.
soldeps-gen reads a dependency's foundry.toml only from GitHub. For a git
dependency hosted elsewhere, the sync fails rather than guess; give it an
override (below) naming where its sources live.
To point a package elsewhere, add a remapping to the root foundry.toml:
[profile.default]
remappings = ["solady/=.turnkey/soldeps/vendor/solady/src/"]
Its prefix must be <name>/ of a declared package (remappings cannot alias
one package under another name), and its target must lie inside
.turnkey/soldeps/vendor/<name>/. tk sync rejects anything else.
The root remappings.txt sits next to foundry.toml and holds every
package's remapping, overrides included, with targets under the soldeps cell
link. Forge resolves targets from foundry.toml's directory, so with the
file at the project root they read:
@openzeppelin/contracts/=.turnkey/soldeps/vendor/@openzeppelin/contracts/
forge-std/=.turnkey/soldeps/vendor/forge-std/src/
With a foundry.toml in a subdirectory, the targets (and the overrides you
write) start with one ../ per level instead. Forge prefers remappings.txt
to the remappings key, and the two agree because the file already contains
the overrides. Don't edit the file; change
foundry.toml and run tk sync.
Native forge
forge build and forge test work in the dev shell alongside tk build and
tk test, the same way running cargo or go natively does. They run from any
directory, since forge finds the root foundry.toml, and cover every package
under src/:
[profile.default]
src = "src"
test = "src"
libs = [".turnkey/soldeps/vendor"]
out = "out"
auto_detect_remappings = false
optimizer = true
optimizer_runs = 200
[fuzz]
runs = 256
src and test are both src, so Solidity code added anywhere under src/ is
covered without editing the file. There is no per-package scoping; narrow a run
with --match-path instead:
forge build
forge test --match-path 'src/contracts/**'
Git-ignore forge's /out/ and /cache/.
Imports resolve through the root remappings.txt that tk sync generates
(see Remappings), into the soldeps cell's vendor/ directory
(libs), where each package's files are linked beside its alias package
(see The Go, Rust and Solidity Cells Are Real Directories). Automatic remapping detection is off, so forge does not guess
remappings of its own.
The compiler comes from the dev shell. foundry.toml sets neither solc
nor solc_version. When Solidity is enabled, the dev shell exports
FOUNDRY_SOLC, the solc the Buck2 toolchain uses, together with
FOUNDRY_OFFLINE=true. Native runs therefore share their compiler with the
Buck2 rules and never download one; the other settings are shared through
foundry.toml itself (see Build Rules). The toolchain
declared in toolchain.toml stays the one place the compiler version is set;
a solc_version in foundry.toml could only go stale on a toolchain bump, so
the pre-commit hook rejects solc and solc_version keys.
Compiler Version
The compiler version comes from the toolchain declared in toolchain.toml
(solidity-toolchain or solc, not both): Buck2's Solidity rules and native forge
(through FOUNDRY_SOLC) both use its solc. To change the version, change the
toolchain.
There is one compiler per repository: the rules take no per-target version.
Building and Testing
# Build contracts
tk build //src/contracts:my_token
# Run tests
tk test //test:my_token_test
# Build all Solidity targets
tk build //... --target-platforms //platforms:solidity
Forge Integration
The solidity_test rule wraps Foundry's forge test, supporting:
- Unit tests
- Fuzz testing
- Fork testing (with
fork_urlattribute) - Gas reports
solidity_test(
name = "integration_test",
srcs = ["Integration.t.sol"],
deps = [":my_token"],
fork_url = "https://eth-mainnet.g.alchemy.com/v2/...", # Optional
)
Jsonnet Support
Turnkey provides Jsonnet support for generating JSON configuration files with Buck2 integration. Jsonnet is a data templating language that extends JSON with variables, functions, and imports.
Setup
Add to toolchain.toml:
[toolchains]
jsonnet = {}
Turnkey uses jrsonnet, a fast Rust implementation of Jsonnet.
Project Structure
my-project/
├── config/
│ ├── base.libsonnet # Shared configuration
│ ├── dev.jsonnet # Development config
│ ├── prod.jsonnet # Production config
│ └── rules.star
Build Rules
jsonnet_library
Compile Jsonnet files to JSON:
load("@prelude//jsonnet:jsonnet.bzl", "jsonnet_library")
jsonnet_library(
name = "config-dev",
srcs = ["dev.jsonnet"],
deps = [":base"], # Dependencies on other jsonnet_library targets
ext_strs = {
"env": "development",
"region": "us-west-2",
},
)
Attributes
| Attribute | Description |
|---|---|
srcs | Jsonnet source files (first file is entry point) |
deps | Dependencies on other jsonnet_library targets |
out | Output filename (defaults to <src>.json) |
ext_strs | External string variables (--ext-str key=value) |
ext_codes | External code variables (--ext-code key=value) |
tla_strs | Top-level argument strings (--tla-str key=value) |
tla_codes | Top-level argument code (--tla-code key=value) |
Example
base.libsonnet
{
// Shared configuration
appName: 'my-app',
version: '1.0.0',
// Environment-specific overrides
envConfig(env):: {
development: {
logLevel: 'debug',
replicas: 1,
},
production: {
logLevel: 'warn',
replicas: 3,
},
}[env],
}
dev.jsonnet
local base = import 'base.libsonnet';
local env = std.extVar('env');
base {
environment: env,
config: base.envConfig(env),
}
rules.star
load("@prelude//jsonnet:jsonnet.bzl", "jsonnet_library")
# Shared library
jsonnet_library(
name = "base",
srcs = ["base.libsonnet"],
)
# Development config
jsonnet_library(
name = "config-dev",
srcs = ["dev.jsonnet"],
deps = [":base"],
ext_strs = {"env": "development"},
)
# Production config
jsonnet_library(
name = "config-prod",
srcs = ["dev.jsonnet"], # Same template, different vars
deps = [":base"],
ext_strs = {"env": "production"},
out = "config-prod.json",
)
Building
# Build a specific config
tk build //config:config-dev
# View the output
tk build //config:config-dev --show-output
cat $(tk build //config:config-dev --show-output 2>&1 | grep -o 'buck-out/[^ ]*')
# Build all configs
tk build //config:...
External Variables
ext_strs (External Strings)
Pass string values from the build system:
jsonnet_library(
name = "config",
srcs = ["config.jsonnet"],
ext_strs = {
"env": "production",
"version": "1.2.3",
},
)
Access in Jsonnet:
{
environment: std.extVar('env'),
version: std.extVar('version'),
}
ext_codes (External Code)
Pass Jsonnet expressions:
jsonnet_library(
name = "config",
srcs = ["config.jsonnet"],
ext_codes = {
"replicas": "3",
"features": "['auth', 'api']",
},
)
Top-Level Arguments
For parameterized configs using functions:
// config.jsonnet
function(env, replicas=1) {
environment: env,
replicas: replicas,
}
jsonnet_library(
name = "config",
srcs = ["config.jsonnet"],
tla_strs = {"env": "production"},
tla_codes = {"replicas": "5"},
)
Use Cases
- Kubernetes manifests - Generate YAML/JSON configs with environment-specific values
- Application configuration - Type-safe config generation with inheritance
- Infrastructure as Code - Generate Terraform JSON, CloudFormation, etc.
- CI/CD pipelines - Generate pipeline configs from templates
CLI Reference
This reference covers all Turnkey CLI commands.
tk - Buck2 Wrapper
tk is the primary Turnkey CLI. It wraps Buck2 with automatic dependency synchronization.
Overview
When using Buck2 with Nix-managed dependencies, certain files must be regenerated when source files change. tk solves this by automatically running sync operations before buck2 commands that read the build graph.
Quick Start
# Use tk just like buck2 - it syncs automatically
tk build //some:target # syncs first, then builds
tk test //some:target # syncs first, then tests
tk run //some:target # syncs first, then runs
# Explicit sync operations
tk sync # manually sync all stale files
tk check # check if files are stale (for CI)
# Skip sync when needed
tk --no-sync build //... # skip sync, run buck2 directly
Command Reference
tk build/run/test/... (Buck2 passthrough)
Most tk commands are passed through to Buck2. Commands that read the build graph automatically sync first:
Sync-first commands (sync before running):
build- Build targetsrun- Run a targettest- Run testsquery- Query the build graphcquery- Configured queryuquery- Unconfigured querytargets- List targetsaudit- Audit the buildbxl- Run BXL scripts
Pass-through commands (no sync):
clean- Clean build artifactskill- Kill Buck2 daemonkillall- Kill all Buck2 processesstatus- Show daemon statuslog- View build logsrage- Generate debug reporthelp- Show helpdocs- Open documentationinit- Initialize a project
Unknown commands default to syncing first (safe default).
tk sync
Explicitly synchronize all stale files, or only those of the named deps
rules (go, rust, pylock, python, javascript, solidity; the
rules are in .turnkey/sync.toml). Rules always run in sync.toml order,
so pylock runs before python, which reads what it writes.
tk sync # sync stale files
tk sync go # sync only go-deps.toml
tk sync --verbose # show what's being synced
tk sync --dry-run # show what would be synced without doing it
Exit codes:
0- Success (files synced or nothing to sync)1- Sync failed
tk check
Check if any files are stale without regenerating them. Useful for CI validation.
tk check # check staleness
tk check rust # check only rust-deps.toml
tk check --verbose # show detailed status
Exit codes:
0- All files up-to-date1- Files are stale (runtk syncto fix)
Example CI usage:
- name: Check files in sync
run: tk check
tk completion
Generate shell completion scripts.
tk completion bash # output bash completion script
tk completion zsh # output zsh completion script
tk completion fish # output fish completion script
Enable completions:
# Bash (add to ~/.bashrc)
eval "$(tk completion bash)"
# Zsh (add to ~/.zshrc)
eval "$(tk completion zsh)"
# Fish (run once)
tk completion fish > ~/.config/fish/completions/tk.fish
tk materialize
Bring deps cells in line with their cell indexes. The shell runs it on every load (direnv's use_turnkey and enterShell), so you rarely need to.
tk materialize /nix/store/…-rustdeps-index.json
It adds missing store links, rewrites alias packages whose target changed, removes what the index no longer names, roots the index under .turnkey/gcroots/, and records the deps file's hash. It fails, and changes nothing, if a store link points anywhere but its own name. A run with nothing to do touches no file.
tk rules
Manage rules.star files that define Buck2 build targets from source files. This command automatically detects imports from source files and updates the deps list in rules.star.
tk rules check # Check every rules.star file against its sources
tk rules sync # Update rules.star files with detected dependencies
tk rules help # Show help
Options:
| Flag | Description |
|---|---|
--all, -a | sync: process all files (skip staleness detection) |
--force, -f | Same as --all |
--verbose, -v | Show detailed output including skipped files |
--quiet, -q | Suppress output |
--dry-run, -n | Show what would be changed without writing |
Staleness Detection:
check always checks every rules.star, so it also catches a stale file that is already committed. sync by default only processes directories with uncommitted changes whose source files are newer than rules.star. Use --all or --force to sync all files.
Examples:
tk rules check # Check all rules.star files
tk rules check src/cmd/tk # Check one directory
tk rules sync # Update stale rules.star files
tk rules sync --all # Force update all files
tk rules sync src/cmd/tk # Sync specific directory
tk rules sync --dry-run # Preview changes without writing
Preserving Manual Dependencies:
If you have manual dependencies that shouldn't be auto-detected, use preservation markers in your rules.star:
# turnkey:preserve-start
"//some/manual:dep",
# turnkey:preserve-end
Dependencies within these markers are preserved during sync.
How tk runs it: tk rules, and the rules sync tk runs before a Buck2
command, call the rules-syncer library (src/rust/rules-syncer) linked
into tk, and print what it reports as above.
tk Flags
Flags must come before the subcommand:
| Flag | Description |
|---|---|
--no-sync | Skip sync and the cell-freshness check, run Buck2 directly |
--no-local | Skip local target overrides from .turnkey/local.toml |
--verbose, -v | Show what tk is doing |
--dry-run, -n | Show what would be synced without doing it |
--quiet, -q | Suppress non-error output |
--help, -h | Show help |
Examples:
tk --no-sync build //... # skip sync
tk --no-local run //target # skip local overrides
tk --verbose sync # verbose sync
tk -v -n sync # dry-run with verbose output
Configuration
Sync Configuration File
tk reads staleness rules from .turnkey/sync.toml. This file is automatically generated from your Nix configuration.
When you configure dependency files in your flake.nix:
goDepsFilegenerates a Go deps rulerustDepsFilegenerates a Rust deps rulepythonDepsFilegenerates a Python deps rule
Example generated sync.toml:
[[deps]]
name = "go"
sources = ["go.mod", "go.sum"]
target = "go-deps.toml"
generator = ["godeps-gen", "--go-mod", "go.mod", "--go-sum", "go.sum"]
[[deps]]
name = "rust"
sources = ["Cargo.toml", "Cargo.lock"]
target_sources = "manifests"
target = "rust-deps.toml"
generator = ["rustdeps-gen", "--cargo-lock", "Cargo.lock", "--platform", "linux-x86_64=x86_64-unknown-linux-gnu", "--platform", "macos-arm64=aarch64-apple-darwin"]
Each [[deps]] entry defines:
name- Human-readable name for this rulesources- Files that trigger regeneration when modifiedtarget- The generated filegenerator- Command to regenerate the targettarget_sources(optional) - A top-level key of the target whose array lists more sources, relative to the project root. They are files only the generator can find, such as a Cargo workspace's member manifests. A target without that key is stale.
Local Target Overrides
tk supports per-developer local overrides via .turnkey/local.toml. This file is not committed to git, allowing each developer to customize target arguments for their local environment.
Use cases:
- Different network addresses for local development
- Debug flags for specific targets
- Custom ports or configuration
Example .turnkey/local.toml:
# Override args for tk run
[run."//docs/user-manual"]
args = ["-n", "192.168.1.100"]
# Override args for tk build
[build."//src/cmd/server:server"]
args = ["--config=debug"]
# Pattern matching with "..."
[test."//src/..."]
args = ["--verbose", "--timeout=60s"]
How it works:
When you run a command that matches a configured target:
tk run //docs/user-manual
# Becomes: buck2 run //docs/user-manual -- -n 192.168.1.100
The args are injected after --, which passes them to the target binary.
Pattern matching:
Patterns ending with ... match any target with that prefix:
//src/...matches//src:foo,//src/pkg:bar,//src/cmd/tool:main//...matches any target
Disable for a single command:
tk --no-local run //docs/user-manual # skips local.toml
Verbose output:
tk --verbose run //docs/user-manual
# Output: tk: applying local override for run //docs/user-manual: [-n 192.168.1.100]
Shell Integration
buck2 alias to tk:
# In devenv shell, buck2 is aliased to tk
buck2 build //... # actually runs: tk build //...
Disable with:
TURNKEY_NO_ALIAS=1 buck2 build //... # uses raw buck2
tw - Native Tool Wrapper
tw wraps native language tools (go, cargo, uv) to keep dependency files in sync when using standard workflows.
The Problem
When you run go get github.com/foo/bar, Go updates go.mod and go.sum. But Buck2 needs go-deps.toml to know about the new dependency. Without auto-sync, you'd need to manually regenerate it.
The Solution
Turnkey transparently wraps go, cargo, and uv so that dependency sync happens automatically:
go get github.com/foo/bar # Just works - go-deps.toml is auto-updated
How It Works
User runs: go get github.com/foo/bar
│
▼
Shell wrapper (provides 'go' binary)
• Sets TURNKEY_REAL_GO to actual go binary path
• Calls: tw go get github.com/foo/bar
│
▼
tw (turnkey wrapper)
1. Loads .turnkey/sync.toml configuration
2. Finds the [[wrappers]] rule for 'go' (none: runs go untouched)
3. Checks if 'get' is a mutating subcommand → yes
4. Captures SHA256 hashes of go.mod, go.sum
5. Runs the real 'go get' command
6. Compares hashes - detects changes
7. Runs the rule's post-commands (go mod tidy)
8. Runs the go deps rule (godeps-gen → go-deps.toml), then any other
rule left stale
Supported Tools
turnkey writes one [[wrappers]] rule per language into
.turnkey/sync.toml, from its language records
(nix/buck2/languages.nix), for each enabled language that has deps rules:
| Tool | Mutating Commands | Watch Files | Deps Rule |
|---|---|---|---|
go | get, mod | go.mod, go.sum | go (go-deps.toml) |
cargo | add, remove, update | Cargo.toml, Cargo.lock | rust (rust-deps.toml) |
uv | add, remove, lock, sync | pyproject.toml, uv.lock | pylock (pylock.toml), then python (python-deps.toml) |
The file names are the configured ones (buck2.go.modFile and so on).
Without buck2.python.uvLockFile, uv watches only pyproject.toml and
runs the python rule.
Escape Hatches
Bypass for a Single Command
TURNKEY_NO_WRAP=1 go get github.com/foo/bar
This runs the real go directly, skipping tw entirely.
Disable Sync for a Command
tw --no-sync go get github.com/foo/bar
This runs through tw but skips the sync step even if files change.
Verbose Output
tw -v go get github.com/foo/bar
Shows what tw is doing:
tw: capturing state of [go.mod go.sum]
tw: detected changes in [go.mod go.sum], running sync
Syncing go-deps.toml...
Running: godeps-gen --go-mod go.mod --go-sum go.sum
Regenerated go-deps.toml
Non-Mutating Commands
Commands not in mutating_subcommands pass through without any overhead:
go build ./... # No snapshot, no sync check - just runs go build
go version # Direct passthrough
Deps generators
tk sync runs a deps generator for each enabled language (the [[deps]]
rules of .turnkey/sync.toml), so you rarely run one yourself. When you do,
every generator takes the same options:
| Option | Description |
|---|---|
-o, --output PATH | Output file (default: stdout) |
--no-prefetch | Skip prefetching the Nix hashes the deps cell fetches with (the file gets placeholder or missing hashes) |
--no-cache | Always fetch from the network, bypassing turnkey's prefetch cache |
Prefetching is on by default. jsdeps-gen takes only --output: the pnpm
lock's integrity hashes are the ones Nix fetches with.
godeps-gen
Generate go-deps.toml from go.mod and go.sum.
Usage
godeps-gen [OPTIONS]
Options
| Option | Description |
|---|---|
--go-mod PATH | Path to go.mod file (default: go.mod) |
--go-sum PATH | Path to go.sum file (default: go.sum) |
--indirect | Include indirect dependencies (default: true) |
Prefetching hashes the modules' proxy.golang.org zips, the source the godeps
cell fetches, in one nix-prefetch-cached --batch call: turnkey's prefetch
cache answers the modules it has seen, and the rest are fetched in parallel.
Examples
# Generate with prefetched hashes
godeps-gen -o go-deps.toml
# Use custom paths
godeps-gen --go-mod src/go.mod --go-sum src/go.sum -o go-deps.toml
# Quick check without fetching (placeholder hashes)
godeps-gen --no-prefetch
rustdeps-gen
Generate rust-deps.toml from Cargo.lock.
Usage
rustdeps-gen [OPTIONS]
Options
| Option | Description |
|---|---|
--cargo-lock PATH | Path to Cargo.lock file (default: Cargo.lock) |
--cargo-toml PATH | Path to the workspace's root Cargo.toml (default: next to Cargo.lock) |
Examples
# Generate from default Cargo.lock
rustdeps-gen -o rust-deps.toml
# Use custom path
rustdeps-gen --cargo-lock rust/Cargo.lock -o rust-deps.toml
pydeps-gen
Generate python-deps.toml from Python dependency files. Each distribution
is recorded as its pure (py3-none-any) wheel, hashed unpacked; a
distribution with only platform-specific wheels, or only an sdist, fails
with its name (see Python).
Usage
pydeps-gen [OPTIONS]
Options
| Option | Description |
|---|---|
--lock PATH | Path to pylock.toml (PEP 751 lock file) - RECOMMENDED |
--pyproject PATH | Path to pyproject.toml |
--requirements PATH | Path to requirements.txt |
--uv-lock PATH | uv.lock, whose dependency graph is recorded with the --lock packages |
--include-dev | Include dev dependencies |
Input Formats
| Format | Flag | Reproducibility | Notes |
|---|---|---|---|
| pylock.toml (PEP 751) | --lock | Best | Exact versions and URLs |
| pyproject.toml | --pyproject | Varies | Uses latest matching versions |
| requirements.txt | --requirements | Varies | Pin versions with == |
Recommended Workflow
# 1. Generate lock file from pyproject.toml
uv lock
# 2. Export to PEP 751 format
uv export --format pylock.toml -o pylock.toml
# 3. Generate python-deps.toml with Nix hashes
pydeps-gen --lock pylock.toml -o python-deps.toml
Examples
# From PEP 751 lock file (best for reproducibility)
pydeps-gen --lock pylock.toml -o python-deps.toml
# From pyproject.toml (resolves to latest matching versions)
pydeps-gen --pyproject pyproject.toml -o python-deps.toml
# From requirements.txt
pydeps-gen --requirements requirements.txt -o python-deps.toml
# Include dev dependencies
pydeps-gen --lock pylock.toml --include-dev -o python-deps.toml
Troubleshooting
"tk: .buckconfig not found"
tk looks for .buckconfig to find the project root. Make sure you're in a Buck2 project directory.
"tk: failed to load sync config"
The .turnkey/sync.toml file is missing or invalid. Ensure you're in a Turnkey project with proper configuration.
"tk: sync failed: generator command failed"
The generator command failed. Check that:
- The generator command is correct
- Required tools are in PATH
- Source files exist
Sync is slow
If sync takes a long time:
- Prefetched hashes are cached (
prefetch-cache.jsonunder$TURNKEY_CACHE_DIR, or elseturnkey/in the platform cache directory, e.g.~/.cache/turnkey/on Linux); check a generator isn't running with--no-cache - Check if generators are doing unnecessary work
Bypass tk
If you need to use raw Buck2:
# Option 1: --no-sync flag
tk --no-sync build //...
# Option 2: TURNKEY_NO_ALIAS environment variable
TURNKEY_NO_ALIAS=1 buck2 build //...
Troubleshooting
Common issues and solutions when using Turnkey.
Shell Issues
"attribute 'X' missing" when entering shell
Cause: A toolchain in toolchain.toml isn't in the registry.
Solution: Either remove the toolchain from toolchain.toml or add it to your registry in flake.nix.
Changes to toolchain.toml not taking effect
Cause: Nix flake caching.
Solution:
- Stage changes:
git add toolchain.toml - Re-enter shell:
exit && nix develop
Build Issues
"toolchain not found" error
Cause: The language toolchain wasn't generated.
Solution: Ensure the toolchain is:
- Declared in
toolchain.toml - Has a mapping in
nix/buck2/mappings.nix(for custom toolchains)
"missing BUCK file" or "missing rules.star"
Cause: Buck2 can't find build files.
Solution: Check that:
.buckconfighas[buildfile] name = rules.star- All cells have proper
.buckconfigwith buildfile settings
Stale dependency errors
Cause: Dependency cells out of sync with lock files.
Solution:
tk sync
tk build //...
Dependency Issues
godeps cell missing packages
Cause: go-deps.toml out of date.
Solution:
godeps-gen > go-deps.toml
git add go-deps.toml
# Re-enter shell
A vendored Rust crate's features differ from what you expect
Cause: Each crate gets the features cargo resolves for cargo test --workspace, per platform, recorded in rust-deps.toml by tk sync. Either
rust-deps.toml is out of date, or a member doesn't ask for the feature.
Solution:
- Regenerate it:
tk sync. A features-only edit to a member'sCargo.tomlis enough to make the Rust rule stale. - Compare with cargo itself:
cargo tree --target <triple> -e normal,dev -i <crate> --format '{p} {f}'. - Ask for the feature in the member's
Cargo.tomlthat uses the crate. There is no override file:rust-features.tomlis retired (see Upgrading).
FUSE Issues
FUSE not available on Linux
Cause: FUSE kernel module not loaded or /dev/fuse missing.
Solution:
# Load FUSE module
sudo modprobe fuse
# Verify
ls /dev/fuse
If persistent, add fuse to /etc/modules-load.d/.
"FUSE-T not installed" on macOS
Cause: FUSE-T package not installed.
Solution:
brew install macos-fuse-t/homebrew-cask/fuse-t
Mount point already in use
Cause: Previous daemon didn't unmount cleanly.
Solution:
# Force unmount
tk compose down --force
# Or manually
fusermount3 -uz /firefly/myproject # Linux
umount -f /firefly/myproject # macOS
"Permission denied" on mount
Cause: User not in fuse group or mount point permissions.
Solution:
# Add user to fuse group (Linux)
sudo usermod -aG fuse $USER
# Log out and back in
# Check mount point permissions
sudo mkdir -p /firefly/myproject
sudo chown $USER:$USER /firefly/myproject
Daemon won't start
Cause: Various issues with daemon lifecycle.
Solution:
# Check for existing processes
pgrep -f turnkey-composed
# Kill stale processes
pkill -9 -f turnkey-composed
# Remove stale socket
rm -f /run/turnkey-composed/*.sock
# Start with debug logging
TURNKEY_FUSE_DEBUG=1 tk compose up
Files appear stale or missing
Cause: Dependency cells updating or policy blocking access.
Solution:
# Check daemon status
tk compose status
# Force refresh
tk compose refresh
# If in "building" state, wait or use lenient policy
TURNKEY_ACCESS_POLICY=lenient tk build //...
"Resource temporarily unavailable" (EAGAIN)
Cause: CI policy returning errors during updates.
Solution:
- Wait for the build to complete
- Switch to development policy for interactive use
- Add retry logic in CI scripts
Build hangs waiting for FUSE
Cause: Strict policy blocking during long Nix builds.
Solution:
# Check what's blocking
tk compose status --verbose
# Use lenient policy for quick iteration
TURNKEY_ACCESS_POLICY=lenient tk build //...
# Or increase timeout
TURNKEY_BLOCK_TIMEOUT=600 tk build //...
Edits not persisting after restart
Cause: Edits stored in overlay, need to generate patches.
Solution:
# Generate patches before stopping
tk compose patch
# Then stop
tk compose down
Container/Docker issues
Cause: FUSE requires privileged access in containers.
Solution:
# Run container with FUSE access
docker run --device /dev/fuse --cap-add SYS_ADMIN ...
# Or disable FUSE and use symlinks
TURNKEY_FUSE_BACKEND=symlink tk build //...
Getting Help
- Check GitHub Issues
- Enable verbose mode:
TURNKEY_VERBOSE=1 nix develop - Check Buck2 logs:
tk log show - FUSE debug logs:
TURNKEY_FUSE_DEBUG=1 tk compose up
Upgrading
What changes when a project moves to a newer turnkey, and what to do about it.
Python distributions are their locked wheels
Each pydeps distribution is now its locked pure (py3-none-any) wheel,
unpacked, instead of its sdist
(ADR 0013).
Its target holds what an installer would put in site-packages: no
setup.py or sdist tests/, src-layout distributions such as requests
import under their own name, and data files and *.dist-info are
resources. Labels are unchanged.
Switching over
- Run
tk sync.python-deps.tomlmoves toschema_version = 3, whoseurlandhashare the wheel's. A schema 2 file fails evaluation and asks fortk sync. - A distribution with no pure wheel fails
pydeps-gen, which names it: one with only platform-specific wheels, or only an sdist. Such a distribution can't be vendored until platform wheels are supported (#248). - User patches against the sdist layout must be regenerated. Paths
change with the layout (
vendor/requests/src/requests/...becomesvendor/requests/requests/...), so such a patch no longer applies and fails its distribution's build. Make the change again in the materialized cell and write the patch withtk compose patch. - Fixups apply to the installed tree. Nothing runs an sdist build, so a fixup that relied on one (or on the sdist's paths) needs rewriting against the wheel's layout.
The per-distribution Python cell
The Python dependency cell is now built one package per distribution, and
laid out by tk materialize as a real directory, .turnkey/pydeps, instead
of one symlink to a merged cell
(ADR 0004,
ADR 0010).
Labels (pydeps//vendor/<name>:<name>) are unchanged. A distribution bump
now rebuilds that distribution alone, plus those whose dependencies name it
if theirs changed, and re-runs only its dependents' actions, with no daemon
restart.
Switching over
- The first shell load after upgrading replaces the
.turnkey/pydepssymlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing. - A lock holding several versions of one distribution fails.
pydeps-gennow rejects a uv resolution that forks on a marker, since the cell holds one version per name: pin the dependency so the lock no longer forks (see Python Workspaces). - User patches move. They go under
.turnkey/patches/pydeps/vendor/<name>/, one directory per distribution, and apply in that distribution's own derivation. A patch file directly under.turnkey/patches/pydeps/fails evaluation with where to move it: move it into the directory of the distribution whose files it changes, keeping its content. A patch spanning two distributions is split into one per distribution.
Rolling back
rm -rf .turnkey/pydeps .turnkey/pydeps.lock .turnkey/gcroots/pydeps
The JavaScript cell's package graph
The JavaScript dependency cell now holds each locked package once, at
vendor/<name>@<version>, and one target per pnpm snapshot, laid out as
pnpm lays out node_modules/.pnpm
(ADR 0012).
A package's own dependencies resolve without the consumer declaring them,
two versions of one name and peer resolutions coexist, and dependency
cycles build. tk materialize lays the cell out as a real directory,
.turnkey/jsdeps, so a package bump re-runs only what depends on it, with
no daemon restart, and plain buck2 reads the new version.
Switching over
- Labels are npm names.
jsdeps//:types_lodashis nowjsdeps//:@types/lodash. Rules sync rewrites the labels it manages; change those in aturnkey:preservesection, or in a target sync doesn't manage, by hand. - Only direct dependencies have labels: the root
package.json's (js-deps.toml's[direct]). A target that listed a package only some dependency needs drops it.@types/...packages are usuallydevDependencies: setbuck2.javascript.includeDevDependencies = true. - Regenerate
js-deps.tomlwithtk sync: the cell needs its[[instance]]and[direct]tables, which need a pnpm 9 lockfile. - The first shell load replaces the
.turnkey/jsdepssymlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. - User patches move under
.turnkey/patches/jsdeps/vendor/<name>@<version>/. A patch file directly under.turnkey/patches/jsdeps/fails evaluation. See Dependency Fixups.
Rolling back
rm -rf .turnkey/jsdeps .turnkey/jsdeps.lock .turnkey/gcroots/jsdeps
The per-package Solidity cell
The Solidity dependency cell is now built one package per Solidity package,
and laid out by tk materialize as a real directory, .turnkey/soldeps,
instead of one symlink to a merged cell
(ADR 0004,
ADR 0011).
Labels (soldeps//:<package>, soldeps//:bundle), the root
remappings.txt and native forge are unchanged. A package bump no longer
restarts the buck2 daemon: it re-runs the Solidity actions, and nothing in
another language. Plain buck2 reads the new version.
Switching over
- The first shell load after upgrading replaces the
.turnkey/soldepssymlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing. - A name declared twice (in
foundry.tomlandpackage.json, or in bothdependenciesanddevDependencies) must resolve to one package:tk syncwrites it once, or fails naming each declaration when they differ. The cell holds one version per name. - User patches move. They go under
.turnkey/patches/soldeps/vendor/<name>/, one directory per package, and apply in that package's own derivation. A patch file directly under.turnkey/patches/soldeps/fails evaluation with where to move it. See Dependency Fixups.
Rolling back
rm -rf .turnkey/soldeps .turnkey/soldeps.lock .turnkey/gcroots/soldeps
The per-module Go cell
The Go dependency cell is now built one package per module, and laid out by
tk materialize as a real directory, .turnkey/godeps, instead of one
symlink to a merged cell
(ADR 0004,
ADR 0008).
Labels (godeps//vendor/<import path>:<last component>) are unchanged. A
module bump now rebuilds that module alone and recompiles only its
dependents: 8 actions for a one-module bump in turnkey's own repo.
Switching over
- The first shell load after upgrading replaces the
.turnkey/godepssymlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing. - User patches move. They go under
.turnkey/patches/godeps/vendor/<module path>/, one directory per module, and apply in that module's own derivation. A patch file directly under.turnkey/patches/godeps/fails evaluation with where to move it: move it into the directory of the module whose files it changes, keeping its content. A patch spanning two modules is split into one per module.
Rolling back
rm -rf .turnkey/godeps .turnkey/godeps.lock .turnkey/gcroots/godeps
The per-crate Rust cell
The Rust dependency cell is now built one package per crate, and laid out by
tk materialize as a real directory, .turnkey/rustdeps, instead of one
symlink to a merged cell
(ADR 0004).
Each crate's features and dependencies come from cargo itself
(ADR 0005,
ADR 0006).
A dependency bump now recompiles only the changed crates' dependents and
re-runs only their tests: about 90 actions for a one-crate bump in turnkey's
own repo, against about 1,200 before.
Switching over
- The first shell load after upgrading regenerates
rust-deps.toml(schema 2, with each crate's package slice) and replaces the.turnkey/rustdepssymlink with the directory.tkrestarts the buck2 daemon once, and the next build is a full one. Nothing else needs doing. tk syncnow runs cargo (cargo treeandcargo metadata, with--locked):Cargo.lockmust match yourCargo.tomlfiles, and the first run downloads the crates into~/.cargo/registry.- Every workspace member's
Cargo.tomlis a source ofrust-deps.toml, so a features-only edit in a member regenerates it.
Rolling back
To go back to an older turnkey, remove the directory before reloading the shell, so that the older shell can create its symlink again:
rm -rf .turnkey/rustdeps .turnkey/rustdeps.lock .turnkey/gcroots/rustdeps
rust-features.toml is retired
buck2.rust.featuresFile no longer exists, and setting it fails evaluation
with a message pointing here. Its overrides are dropped: each crate gets the
features cargo resolves for your workspace.
- To get a feature, ask for it in the
Cargo.tomlof the member that uses the crate, for exampleserde = { version = "1", features = ["derive"] }. - Then remove the option and delete
rust-features.toml.
[[requested]] is gone from rust-deps.toml too; regenerating it removes it.
Features may differ slightly
The old resolver was turnkey's own; the new one is cargo's. In turnkey's own
repo, four of 317 crates changed, all to what cargo build does:
- a crate no longer gets dependencies that only apply on targets you don't
build (
jiff'sportable-atomic); - features that only one platform enables are set on that platform only
(
mio'slog,zerocopy'sderive); - features no configured platform enables are gone (
tokio'swindows-sys).
A crate that no configured platform builds (Windows-only, wasm, a build dependency) gets no features or dependencies, since nothing configures it.
User patches
Patches that tk compose patch writes for the Rust cell now live in their
crate's directory, .turnkey/patches/rustdeps/vendor/<crate>@<version>/, and
apply in that crate's own package. A patch file left directly under
.turnkey/patches/rustdeps/ fails evaluation: move it into its crate's
directory, or regenerate it with tk compose patch. See
Dependency Fixups.