👽ZK-Node

Authors: [man4ela | catapulta.eth]

System Requirements

CPU	OS	RAM	DISK
4+ cores CPU	Debian 12/Ubuntu 22.04	=> 16 GB RAM	1TB+ (SSD or NVMe)

The X Layer archive node has a size of 560GB on November 18th, 2024

Setup XLayer Node

X Layer is an EVM-compatible Layer 2 network built with Polygon CDK, using Zero-Knowledge (ZK) technology to enhance Ethereum’s scalability, security, and efficiency

This guide covers the installation of X Layer Node (referred to as ZKNode), Synchronizer, ZKProver, and configuration of the State and Pool databases

NOTE: You will need Ethereum L1 RPC endpoint in order to sync X Layer

With their Eggfruit upgrade in September 2024, the Polygon team made the official recommendation that all infra providers will need to begin running the cdk-erigon RPC Node. While zkEVm Node is still operational, it is no longer being maintained by the Polygon team. Additionally, we found slight POI divergence in our integration testing. Please use the test below as an initial screening when working with this chain.

Indexer Test for X Layer POI

• Sync the following subgraph: QmWHYMV9mPZ6zoomwWSZbN24sdGSEQhy1efritMiETpxqS

• Query to grab the normalized POI

  • Note: Do not modify the query below - the indexer address in the query should be the 0x0 address.

{
  "query": "{ proofOfIndexing(subgraph: \\"QmWHYMV9mPZ6zoomwWSZbN24sdGSEQhy1efritMiETpxqS\\", blockNumber: 3041190, blockHash: \\"0xa819924ad94bcf3295826d5ad916c9ef06fac8cb46a6273d3bcc7aec822e22e7\\", indexer: \\"0x0000000000000000000000000000000000000000\\") }"
}

If there is a match for the Consensus POI provided below, it may indicate that the setup allows for syncing other subgraphs and being in majority consensus. If there is a match for the Divergent POI, this can be an indication of a data determinism issue.

Consensus 0xa1223b5cbabf16d9896c2bd19099d08e5ce45c7ff308674b3ea7ada5367334bf

Divergent 0x411bf0293e96a1459167ef1828aa7d70cd6c2e1f8c4210e0edf0fa8827eeed69

• Shell into the index node and run this curl command curl -s -X POST -H "Content-Type: application/json" --data '{"query": "{ proofOfIndexing(subgraph: "QmWHYMV9mPZ6zoomwWSZbN24sdGSEQhy1efritMiETpxqS", blockNumber: 3041190, blockHash: "0xa819924ad94bcf3295826d5ad916c9ef06fac8cb46a6273d3bcc7aec822e22e7", indexer: "0x0000000000000000000000000000000000000000") }"}' "http://localhost:8030/graphql"

Pre-Requisties

sudo apt update -y && sudo apt upgrade -y && sudo apt autoremove -y

sudo apt install build-essential libbenchmark-dev libomp-dev libgmp-dev nlohmann-json3-dev postgresql libpqxx-dev libpqxx-doc nasm libsecp256k1-dev grpc-proto libsodium-dev libprotobuf-dev libssl-dev cmake libgrpc++-dev protobuf-compiler protobuf-compiler-grpc uuid-dev

Important dependency note: You must install libpqxx version 6.4.5. If your distribution installs a newer version, please compile libpqxx 6.4.5 and install it manually instead

Use dpkg to check the Installed version of libpqxx:

dpkg -l | grep libpqxx

Setting up Firewall

Set explicit default UFW rules

sudo ufw default deny incoming
sudo ufw default allow outgoing

Allow SSH

sudo ufw allow 22/tcp

Allow remote RPC connections with Blast Node

sudo ufw allow from ${REMOTE.HOST.IP} to any port 8545

sudo ufw allow 5432/tcp

Not advised to allow all or unknown IP address to RPC port

Enable Firewall

sudo ufw enable

To check the status of UFW and see the current rules

sudo ufw status verbose

Install GO

Go version 1.21+ is required to build ZKnode

sudo wget https://go.dev/dl/go1.21.6.linux-amd64.tar.gz && \
sudo rm -rf /usr/local/go && \
sudo tar -C /usr/local -xzf go1.21.6.linux-amd64.tar.gz && \
rm go1.21.6.linux-amd64.tar.gz
​
echo 'export PATH=$PATH:/usr/local/go/bin:/root/.local/bin' >> /root/.bashrc
​
source /root/.bashrc
​
#verify Go installation
go version

Build ZKnode

mkdir && cd zknode

git clone --recurse-submodules https://github.com/okx/xlayer-node.git

cd xlayer-node

make build

Move the xlayer-node binary from the build directory to the designated directory for execution:

mkdir -p /root/zknode/{node,prover}

mv /root/zknode/xlayer-node/dist/xlayer-node /root/zknode/node/

Download ZKnode configuration and genesis.json file:

mkdir -p /root/zknode/node/config

wget https://raw.githubusercontent.com/okx/Deploy/main/setup/zknode/mainnet/config/node.config.toml -O /root/zknode/node/config/node.config.toml

wget https://raw.githubusercontent.com/okx/Deploy/refs/heads/main/mainnet/genesis.config.json -O /root/zknode/node/config/genesis.config.json

Carefully change the following parameters inside configuration file:

 sudo nano /root/zknode/node/config/node.config.toml

[State.DB]

Host = "xlayer-mainnet-state-db"

#specify Host = "127.0.0.1"

[Pool.DB]

Host = "xlayer-mainnet-pool-db"

#specify Host = "127.0.0.1"

[Etherman]

URL = ""

#input your synced L1 Ethereum RPC endpoint

[Synchronizer]

SyncInterval = "1s"

SyncChunkSize = 100

#add a line L1SynchronizationMode = "sequential"

[MTClient]

URI = "xlayer-mainnet-prover:50061"

#specify URI = "127.0.0.1:50061"

[Executor]

URI = "xlayer-mainnet-prover:50071"

#specify URI = "127.0.0.1:50071"

[Metrics]

Host = "0.0.0.0"

#specify Host = "127.0.0.1"

[HashDB]

Host = "xlayer-mainnet-state-db"

#specify Host = "127.0.0.1"

ctrl X + Y and enter #to save changes

Compile and build ZKProver

cd /root/zknode/

git clone --recursive https://github.com/0xPolygonHermez/zkevm-prover.git

cd zkevm-prover

Download necessary files

Download an archive (~75GB). It's a good idea to have it running in the screensession:

./tools/download_archive.sh

The archive will take up an additional 115GB of space once extracted.

Consider to recompile the protobufs:

cd src/grpc
make
cd ../..

Run make to compile the main project:

make clean
make generate
make -j

The compilation process may appear to be stuck at some point, but sit back, relax, and wait till it completes the building

Test Vectors

./build/zkProver -c testvectors/config_runFile_BatchProof.json

Download configuration files:

wget -P /root/zknode/prover https://raw.githubusercontent.com/okx/Deploy/main/setup/zknode/mainnet/config/prover.config.json

Modify prover.config.json:

sudo nano /root/zknode/prover/prover.config.json

Replace

"databaseURL": "postgresql://prover_user:prover_pass@xlayer-mainnet-state-db:5432/prover_db",

with

"databaseURL": "postgresql://prover_user:prover_pass@127.0.0.1:5432/prover_db",

Create systemd files for the ZKEVM Node components

Zknode:

sudo nano /etc/systemd/system/xlayer-rpc.service

Paste the configs and save by entering ctrl+X and Y+ENTER:

[Unit]
Description=XLayer RPC Service
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=simple
Restart=on-failure
RestartSec=5
TimeoutSec=900
User=root
Nice=0
LimitNOFILE=200000
WorkingDirectory=/root/zknode/node/
ExecStart=/root/zknode/node/xlayer-node run \
   --network custom \
   --custom-network-file /root/zknode/node/config/genesis.config.json \
   --cfg /root/zknode/node/config/node.config.toml \
   --components rpc

KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Synchronizer:

sudo nano /etc/systemd/system/xlayer-sync.service

[Unit]
Description=XLayer Sync Service
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=simple
Restart=on-failure
RestartSec=5
TimeoutSec=900
User=root
Nice=0
LimitNOFILE=200000
WorkingDirectory=/root/zknode/node/
ExecStart=/root/zknode/node/xlayer-node run \
   --network custom \
   --custom-network-file /root/zknode/node/config/genesis.config.json \
   --cfg /root/zknode/node/config/node.config.toml \
   --components synchronizer

KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

and ZKProver:

sudo nano /etc/systemd/system/xlayer-prover.service

[Unit]
Description=XLayer Prover Service
After=network.target
StartLimitIntervalSec=60
StartLimitBurst=3

[Service]
Type=simple
Restart=on-failure
RestartSec=5
TimeoutSec=900
User=root
Nice=0
LimitNOFILE=200000
WorkingDirectory=/root/zknode/zkevm-prover/
ExecStart=/root/zknode/zkevm-prover/build/zkProver -c /root/zknode/prover/prover.config.json

KillSignal=SIGTERM

[Install]
WantedBy=multi-user.target

Configure Postgresql for ZKProver

Make sure to be inside /root/zknode/zkevm-prover/ directory:

./tools/statedb/create_db.sh prover_db prover_user prover_pass

Example of expected output

ProverDB database creation
Installing PostgreSQL...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
postgresql is already the newest version (14+238).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Creating database prover_db...
could not change directory to "/root/zknode/zkevm-prover": Permission denied
Creating user prover_user...
could not change directory to "/root/zknode/zkevm-prover": Permission denied
CREATE ROLE
could not change directory to "/root/zknode/zkevm-prover": Permission denied
GRANT
Creating table state.merkletree...
CREATE SCHEMA
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
CREATE TABLE
INSERT 0 1
Done.
Example of connection string to use in the config.json file:
  "databaseURL": "postgresql://prover_user:prover_pass@127.0.0.1:5432/prover_db"

Depending on your Postgresql version (14 or 15) update postgresql.conf:

sudo nano /etc/postgresql/14/main/postgresql.conf

Set max_connections = 500

Save, Quit and reload postgresql to apply changes by running:

sudo systemctl restart postgresql

Launch X Layer node:

sudo systemctl daemon-reload

sudo systemctl start xlayer-prover
sudo systemctl start xlayer-rpc.service
sudo systemctl start xlayer-sync.service

Monitor logs

sudo journalctl -fu xlayer-prover -n 250

sudo journalctl -fu xlayer-rpc.service -n 250

sudo journalctl -fu xlayer-sync.service -n 250

During the synchonization, you are expected to get following log messages:

ZKNode:

Nov 19 06:19:24 xlayer-node[3963121]: 2024-11-19T06:19:24.558Z        INFO        state/l2block.go:280        new l2 block detected: number 43801, hash 0x697fcd84070802e874a83723d16a760fcf544bab23b9c2c3c1e8047b485b1c47        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:19:24 xlayer-node[3963121]: 2024-11-19T06:19:24.559Z        INFO        state/l2block.go:280        new l2 block detected: number 43802, hash 0x3fc86022298cdeafd950a55dba81bc30ac06d52c73c43298180839374f68964f        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:19:24 xlayer-node[3963121]: 2024-11-19T06:19:24.559Z        INFO        state/l2block.go:280        new l2 block detected: number 43803, hash 0xcec7e916fb50f140a3573e491df07369b309cbcc5120577bc366765b5dd4b039        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:19:24 xlayer-node[3963121]: 2024-11-19T06:19:24.586Z        INFO        jsonrpc/dynamic_gas_price_xlayer.go:84        Dynamic gas price update period is 10s        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:19:34 xlayer-node[3963121]: 2024-11-19T06:19:34.585Z        WARN        pool/pool.go:647        No suggested min gas price since: 2024-11-19 06:14:34.585576643 +0000 UTC        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:19:34 xlayer-node[3963121]: github.com/0xPolygonHermez/zkevm-node/pool.(*Pool).pollMinSuggestedGasPrice
Nov 19 06:19:34 xlayer-node[3963121]:         /root/zknode/xlayer-node/pool/pool.go:647
Nov 19 06:19:34 xlayer-node[3963121]: github.com/0xPolygonHermez/zkevm-node/pool.(*Pool).StartPollingMinSuggestedGasPrice.func1
Nov 19 06:19:34 xlayer-node[3963121]:         /root/zknode/xlayer-node/pool/pool.go:181
Nov 19 06:19:34 xlayer-node[3963121]: 2024-11-19T06:19:34.587Z        INFO        jsonrpc/dynamic_gas_price_xlayer.go:84        Dynamic gas price update period is 10s        {"pid": 3963121, "version": "v0.4.1-2-g837ab57e"}

ZKProver:

Nov 19 05:11:49 zkProver[3961854]: 20241119_051149_849293 aa38631 4252c40 --> BINARY_EXECUTOR starting...
Nov 19 05:11:49 zkProver[3961854]: 20241119_051149_849299 aa38631 4252c40 --> BINARY_BUILD_FACTORS starting...
Nov 19 05:11:50 zkProver[3961854]: 20241119_051150_983649 aa38631 4252c40 <-- BINARY_BUILD_FACTORS done: 1.134332 s
Nov 19 05:11:50 zkProver[3961854]: 20241119_051150_983665 aa38631 4252c40 --> BINARY_BUILD_RESET starting...
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_169003 aa38631 4252c40 <-- BINARY_BUILD_RESET done: 0.185324 s
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_169019 aa38631 4252c40 <-- BINARY_EXECUTOR done: 1.319726 s
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_169102 aa38631 4252c40 <-- PROVER_CONSTRUCTOR done: 6.762991 s
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_169105 aa38631 4252c40 Launching HashDB server thread...
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_169150 aa38631 4252c40 Launching executor server thread...
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_170116 aa38631 37fe640 Executor server listening on 0.0.0.0:50071
Nov 19 05:11:51 zkProver[3961854]: 20241119_051151_170129 aa38631 3fff640 HashDB server listening on 0.0.0.0:50061

Synchronizer:

Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.713Z        INFO        synchronizer/synchronizer.go:701        Position: 8. New block. BlockNumber: 19549461. BlockHash: 0xfc7aeddc5a22c867253bde2c2dcb54ba3fc971b2d0ff60564944747db27608b8        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.713Z        INFO        synchronizer/synchronizer.go:627        Syncing block 19549461 of 21219852        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.713Z        INFO        synchronizer/synchronizer.go:628        Getting rollup info from block 19549461 to block 19549562        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.735Z        INFO        etherman/etherman_xlayer.go:1239        tx hash: 0x95b701bc4bad615ce9790bac36273f56f5b5a0404dd4384dcb7cabd60ad72496, msg form:0xAF9d27ffe4d51eD54AC8eEc78f2785D7E11E5ab1, to:0x2B0ee28D4D51bC9aDde5E58E295873F61F4a0507        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.883Z        INFO        etherman/etherman_xlayer.go:1239        tx hash: 0x40d7c22c8f0c9087778c392e9d41f31b058dfa355f55579153d45bc03e53f9b5, msg form:0xAF9d27ffe4d51eD54AC8eEc78f2785D7E11E5ab1, to:0x2B0ee28D4D51bC9aDde5E58E295873F61F4a0507        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}
Nov 19 06:13:10 xlayer-node[3963058]: 2024-11-19T06:13:10.885Z        INFO        dataavailability/dataavailability.go:73        Try to get data from trusted sequencer for batches [14]        {"pid": 3963058, "version": "v0.4.1-2-g837ab57e"}

Run curl command in the terminal to check the status of your node

curl -H "Content-type: application/json" -X POST --data '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}' http://localhost:8545

When it returns false then your node is fully synchronized with the network

References

GitHub - okx/Deploy: Deploy script for X Layer both mainnet and testnetGitHub

xlayer-node/config/environments/mainnet at dev · okx/xlayer-nodeGitHub

Setup local ZKNode | X Layer DocumentationOKX