Dev scripts¶
Running and testing the agent¶
../dev/test_agent.py: Singleshot, barebones debug¶
A barebones script to test the agent. Beware, unlike a more full-featured GUI or TUI, this is a single-shot test of the agent with no intermediate streaming output; it will only run one command, will display no intermediate output, and then will dump output at the end.
This should be used for sanity testing and simple debugging only.
How to run, example:
$: python test_agent.py "Submit an elastic DFT workflow for GaAs."
../dev/langgraph_server.sh: Interactive server¶
Run an interactive langgraph server to debug the agent.
Select the agent by changing ../dev/langgraph.json.
For the most part, you will want to use ../dev/serve.py, which builds the agent with
create_atomatic_agent_nockpt and hands checkpointing off to the langgraph dev
server.
Details for nerds, click to expand
The server holds the threads in memory and writes them to ../dev/.langgraph_api
only at a clean shutdown, so ../dev/serve.py writes them every 10 seconds instead. A
crash then loses at most 10 seconds of a conversation.
../dev/langgraph_server.sh starts the server with four jobs per worker. With the
default of one, a tool call that waits (a poll of a job queue, for example)
holds the only worker, and every other thread queues behind it with no way to
send a message or to cancel.
The server also keeps the runs of the last session in ../dev/.langgraph_api. A
run that did not finish stays pending, and at the next start the queue takes
it and repeats an agent turn that you cancelled or abandoned. ../dev/serve.py
releases those runs at import, before the queue starts, and sets their threads
back to idle. Look for released N run(s) of the last server in the log.
../dev/.langgraph_api is a cache of pickle files. The server deletes all of it
when a change to this repository makes it unreadable, so ../dev/serve.py also copies
each new checkpoint into server_checkpoints.db at the repository root. Nothing
deletes that archive. To read a thread back from it:
import sqlite3
from deepagents import create_deep_agent
from langgraph.checkpoint.sqlite import SqliteSaver
from atomatic.harness.agent import CONFIG
saver = SqliteSaver(sqlite3.connect("server_checkpoints.db"))
agent = create_deep_agent(**(CONFIG | {"checkpointer": saver}))
state = agent.get_state({"configurable": {"thread_id": "<thread id>"}})
print(state.values["messages"])
How to run, example:
$: ./langgraph_server.sh
Ablating skills¶
../dev/ablate_deepagent.py¶
Run an ablation study of skills readable by the deepagent. This copies files
into the "active skills" (skills-active) directory which is read by the atomatic
agent.
Run --help for usage.
../dev/ablate_other_harness.py¶
Run an ablation study of skills readable by other harnesses. This is literally
just copying files into your directory of choice, e.g., /path/to/.opencode or
similar.
Run --help for usage.
Other info¶
Guiding principles for skills¶
- As few skills as necessary,
- Each skill as modular as possible,
- With broader skills covering general cases,
- And specific ones refining them,
- With all skills containing only the most essential info.