Running an Espresso Node
Information on the different ways to run an Espresso node. The Espresso node is referred to as a 'sequencer node' from here on forth.
Basic Usage
Visit the espresso-sequencer repository for instructions on how to run a sequencer node natively or with docker. Find the latest docker images here.
Usage:
For brevity, we will omit the full sequencer path from here and simply refer to the executable as sequencer
going forward.
The sequencer docker compose file is a great reference point for configuring an entire local sequencer network, including a few sequencer nodes, an L1 client, and the HotShot data availability and orchestrator servers.
Required Parameters
Consensus State
These required parameters define the initial consensus state. All nodes in a given network must use the same values for these parameters.
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_CHAIN_ID | --chain-id | Unique numeric identifier for this network |
ESPRESSO_SEQUENCER_MAX_BLOCK_SIZE | --max-block-size | Maximum size of a block. Can be specified with units like |
ESPRESSO_SEQUENCER_BASE_FEE | --base-fee | Base fee in WEI per byte sequenced |
ESPRESSO_SEQUENCER_PREFUNDED_BUILDER_ACCOUNTS | --prefunded-builder-accounts | (testnet only) Comma-separated list of prefunded accounts for paying sequencing fees |
Network
These required parameters determine how the node connects to the network and identifies itself.
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_L1_PROVIDER | --l1-provider-url | JSON-RPC URI of the L1 provider (e.g. http://localhost:8545. |
ESPRESSO_SEQUENCER_ORCHESTRATOR_URL | --orchestrator-url | URL of the HotShot orchestrator. This service will be provided by Espresso. |
ESPRESSO_SEQUENCER_CONSENSUS_SERVER_URL | --consensus-server-url | URL of the HotShot consensus web server. This service will be provided by Espresso. |
ESPRESSO_SEQUENCER_DA_SERVER_URL | --da-server-url | URL of the HotShot DA web server. This service will be provided by Espresso. |
ESPRESSO_SEQUENCER_STATE_RELAY_SERVER_URL | --state-relay-server-url | URL of the state relay web server. This service will be provided by Espresso. |
ESPRESSO_SEQUENCER_STATE_PEERS | --state-peers | Comma-separated list of peer URLs to use for catchup. This may include the archival query service operated by Espresso as well as URLs of your own nodes (see catchup) |
ESPRESSO_SEQUENCER_KEY_FILE | --key-file | Path to file containing private signing keys. See key management. |
Optional Parameters
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_PRIVATE_STAKING_KEY | --private-staking-key | The private staking key to use for signing consensus messages. This parameter replaces the required |
ESPRESSO_SEQUENCER_PRIVATE_STATE_KEY | --private-state-key | The private key to use for signing finalized consensus states. See also |
Optional Modules
The sequencer node supports a set of optional modules that extend the node with useful APIs (e.g. transaction submission or query functionality). Here we describe what these modules do and how to enable them.
In general, modules are enabled in the following way:
HTTP
This module runs a basic HTTP server that comes with healthcheck and version endpoints. Additional endpoints can be enabled with the modules listed below.
Usage:
Parameters:
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_API_PORT | --port | Port that the HTTP API will use. |
Status
This module extends the HTTP API with telemetry and consensus metrics (e.g. an endpoint to retrieve the latest block height).
Usage:
This will add a Prometheus endpoint GET /status/metrics
containing useful metrics for monitoring the performance of the node and network. See also monitoring.
Catchup
This module extends the HTTP API with a module that serves queries for pending consensus state. Other nodes can connect to this API to quickly sync with the latest state in the event that they fall out of sync with consensus.
Usage:
Query
This module enables a HotShot query service API that connects to a persistent store containing the history of the blockchain. This API provides endpoints that rollups can use to integrate with the sequencer.
This module must be enabled alongside the http
module. The query API can be accessed at the port specified by the http
module. This option also requires a storage module, which defaults to storage-fs
(see below for more details on storage options).
Usage:
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_API_PEERS | --peers | A comma-separated list of peer query service URLs to fetch missing data from. |
Filesystem Storage
This module enables a local storage backend for the query service and consensus state. Eventually, the backend will also store DA blocks and VID shares. This setting is useful for testing and debugging, but is not recommended for production nodes because it is currently not very stable or performant. Long term, we hope to improve this storage option.
Usage:
Parameters:
Env Variable | CLI Flag | Description |
---|---|---|
ESPRESSO_SEQUENCER_STORAGE_PATH | --path | Storage path for persistent data. |
SQL Storage
This module enables a postgres storage backend for the query service and consensus state. Eventually, the backend will also store DA blocks and VID shares. This setting is recommended for production nodes.
Usage:
Parameters:
Env Variable | CLI Flag | Description |
---|---|---|
N/A | --uri | This is a shorthand for setting a number of other options all at once in URI form. Components of this URI can overridden by the parameters below. Example: postgres[ql]://[username[:password]@][host[:port],]/database[?parameter_list]. |
ESPRESSO_SEQUENCER_POSTGRES_HOST | --host | Hostname for the remote Postgres database server. |
ESPRESSO_SEQUENCER_POSTGRES_PORT | --port | Port for the remote Postgres database server. |
ESPRESSO_SEQUENCER_POSTGRES_DATABASE | --database | Name of database to connect to. |
ESPRESSO_SEQUENCER_POSTGRES_USER | --user | Postgres user to connect as. |
ESPRESSO_SEQUENCER_POSTGRES_PASSWORD | --password | Password for Postgres user. |
ESPRESSO_SEQUENCER_POSTGRES_USE_TLS | --use-tls | Use TLS for an encrypted connection to the database. |
ESPRESSO_SEQUENCER_POSTGRES_PRUNE | --prune | Use this flag or set variable to |
ESPRESSO_SEQUENCER_PRUNER_PRUNING_THRESHOLD | --pruning-threshold | If storage usage exceeds this threshold (in bytes), data younger than the target retention will be pruned (possibly up to the minimum retention) |
ESPRESSO_SEQUENCER_PRUNER_MINIMUM_RETENTION | --minimum-retention | The minimum time which data must be retained, regardless of storage usage |
ESPRESSO_SEQUENCER_PRUNER_TARGET_RETENTION | --target-retention | The desired amount of time to retain data, storage permitting |
ESPRESSO_SEQUENCER_PRUNER_BATCH_SIZE | --batch-size | The number of objects to delete at once when pruning |
ESPRESSO_SEQUENCER_PRUNER_MAX_USAGE | --max-usage | The maximum fraction of pruning-threshold to use. If storage usage exceeds pruning-threshold, it will be pruned back to this fraction of pruning-threshold. Expressed as an integer on a scale of 1 to 10000. |
ESPRESSO_SEQUENCER_PRUNER_INTERVAL | --internval | Interval for running the pruner. |
Submit
This module extends the HTTP API with a POST endpoint to submit a transaction for sequencing.
Usage:
Key Management
Each sequencer node needs two signing key pairs to run:
The staking key is a BLS key used to sign consensus messages (votes, proposals), and it supports efficient signature aggregation, important for consensus performance.
The state key is a Schnorr key used to sign finalized consensus states, which in turn drives the onchain Espresso light client on L1.
These keys are typically stored in a .env
file within the sequencer container, and the sequencer is configured via ESPRESSO_SEQUENCER_KEY_FILE
to load private keys from this file. While you are welcome to generate these keys however you like, as long as they have the right performance, Espresso provides a utility program keygen
which is distributed with the sequencer
image. The simplest way to generate keys is:
Here $CONTAINER
is the ID of the Docker container in which you will be running the sequencer
node (built from the ghcr.io/espressosystems/espresso-sequencer/sequencer:main
image). This command will generate a file called /keys/0.env
in the Docker container containing the private keys. It will also print the generated public keys in the terminal. You can then pass this file to the sequencer by setting ESPRESSO_SEQUENCER_KEY_FILE=/keys/0.env
.
This method of generating keys is nice because the keys never leave the Docker container where they will be used. If, however, you want to store the keys on the host machine as well, or if you want to use a different container to generate the keys than you will use to run the sequencer (such as a one-off container from docker run
) you need only create a Docker volume to store the keys in a host directory, such as
This will store the generated keys at ./keys
on the host. You can then pass them into the sequencer by mounting the same volume in the sequencer container.
The keygen
utility has some additional options which you can view by running with --help
. One of the most useful is --seed <SEED>
, to use a seed for generating the keys deterministically, instead of using entropy from the OS. This is particularly useful if you want to use your own entropy instead of the default entropy source: you can generate a randomized seed however you like and then pass it to the keygen
program. The seed is a 32-byte integer encoded as hex (with no 0x
prefix).
Monitoring
When running the status API, the performance of a sequencer node can be monitored using Prometheus tools, by monitoring the endpoint /status/metrics
. Some of the most important metrics to monitor include:
consensus_current_view
: should be incrementing once every 1-2 seconds. In rare cases it is acceptable for this metric to remain static for up to 1 minute. If not increasing, it may mean the network has lost liveness, or your node has fallen out of sync with the network.consensus_last_decided_view
: should be increasing mostly in tandem withconsensus_current_view
. Ifcurrent_view
is increasing butlast_decided_view
is not, it indicates a network-wide problem with consensus state, or a recurring problem with builders or proposers.consensus_outstanding_transactions
: smaller is better, and this metric should not show any trend over time. If it is especially large (relative to volume) or increasing over a long period of time, it may indicate that your node is not garbage collecting the public mempool properly.
Modes
It is possible to run an Espresso node in three modes, differentiated by how long the nodes store historical data.
Lightweight Node
A lightweight node stores only the data needed to run consensus. It does not keep any historical data, and it is not eligible to be on the consensus DA committee. It has negligible storage requirements on the order of kilobytes. A lightweight node is any node running without the optional query module.
Archival Node
An archival node stores all historical data in perpetuity, and is thus able to serve queries for arbitrary historical state. It is eligible to be on the DA committee. Its storage requirements depend on how much data the network is processing, which in turn depends on how much the network is being used. In testnets, this has been on the order of tens of gigabytes per month, but in mainnet this may be more.
To run an archival node, simply enable the optional query module without any of the pruning options.
DA Node
Pruning of old data is not yet supported with filesystem storage, and thus it is only possible to run a DA node with Postgres storage at this time.
A DA node provides data availability for recently finalized data. It is eligible to be one the DA committee, because it will make data available for long enough for an archival node to fetch it and persist it, before the data is pruned from the DA node. Unlike the archival node, the DA node has bounded storage requirements.
The storage requirements for a DA node are determined by how long we want it to retain data in the worst case. Typically, we want DA nodes to retain data for 1 week under average load, and a minimum of 1 day under worst case load, which give archival nodes time to ensure the data is persisted long term. Detailed hardware requirements for DA nodes are given below.
To run a DA node, enable the optional query module as you would for an archival node, but additionally set pruning parameters:
ESPRESSO_SEQUENCER_POSTGRES_PRUNE=true
ESPRESSO_SEQUENCER_PRUNER_MINIMUM_RETENTION=1d
ESPRESSO_SEQUENCER_PRUNER_TARGET_RETENTION=7d
ESPRESSO_SEQUENCER_PRUNER_PRUNING_THRESHOLD
set to the worst case storage usage, in bytes, based on the hardware requirements
Hardware Requirements
Hardware requirements are still in flux as we refine our testnets and add new features, but for now we recommend the following:
RAM: 16-32 GB.
CPU: 2-4 Cores.
Storage (DA node): 20 GB minimum, ability to scale to 1 TB on demand.
Storage (non-DA Node): Negligible, kilobytes
Last updated