dcrubro/skalacoin
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: dcrubro/skalacoin:nodediscovery
Branches Tags
dcrubro/skalacoin:master dcrubro/skalacoin:nodediscovery dcrubro/skalacoin:txtest dcrubro/skalacoin:p2ptest
...
pull from: dcrubro/skalacoin:master
Branches Tags
dcrubro/skalacoin:master dcrubro/skalacoin:nodediscovery dcrubro/skalacoin:txtest dcrubro/skalacoin:p2ptest
21 Commits
nodediscovery ... master
Author SHA1 Message Date
dcrubro f40ffaa6f7 verbosity 2026-08-16 23:23:32 +02:00
dcrubro d7bcd64130 Fix analyzer conflicts 2026-08-16 23:15:57 +02:00
dcrubro af888258f4 cmake analyzer and strict modes 2026-08-16 22:08:47 +02:00
dcrubro 914fa6e5a7 Change cmakelists for compile strictness 2026-08-16 20:48:24 +02:00
dcrubro 309367a91e Add sendrawtx command, specify send command to not broadcast 2026-08-03 17:28:12 +02:00
dcrubro 10d5d71a9f Prevent transaction replay and stop reorgs from destroying transactions 
Two gaps in the account model, found while working out how many confirmations
are actually needed for a payment to be safe.

== A reorg silently destroyed transactions ==

chain.c removed transactions from the mempool when they were mined but never
put them back. TxMempool_Insert had exactly two callers -- the `send` command
and inbound broadcasts -- so a transaction that existed only in an orphaned
block was gone from both the chain and the pool, and would never be mined
unless its sender happened to rebroadcast.

That makes the usual reassurance about accidental orphans ("your transaction
just lands in the next block") false for this node. Chain_RollbackToHeightLocked
now returns the discarded blocks' non-coinbase transactions to the mempool
before anything is freed, going through the existing Chain_BorrowBlockTransactions
disk fallback because those blocks are usually header-only by then.

Coinbases are deliberately not restored: they are bound to a specific height and
reward and are invalid anywhere else.

A transaction that also appears in the replacing branch needs no special
handling. Chain_ReplaceBranch rolls back and applies under a single lock
acquisition, and Chain_AddBlockLocked removes every applied transaction from the
pool again -- so it is re-inserted and removed moments later, with no window in
which a miner could pick up a copy of something already back in the chain. That
ordering is what makes this safe in an account model, where re-mining a
transaction would debit the sender twice, so it is spelled out at the site.

== Any historical transaction could be replayed ==

Nothing rejected a transaction whose hash was already in the chain.
Block_AllTransactionsValid checks signatures and that there is exactly one
coinbase; it never looked for repeats, and the mempool's hash-keyed dedup is
mempool-only. So anyone could take a mined, publicly visible signed transaction,
rebroadcast it, and have it mined again -- debiting the sender a second time.
UTXO chains get this for free because the spent inputs no longer exist; an
account model has nothing to stop it.

balance_sheet_entry_t gains lastTxTimestamp, and a non-coinbase transaction is
now valid only if its timestamp is strictly greater than its sender's last
included one. Transaction timestamps are unix MILLISECONDS (the comment in
transaction.h claimed seconds and was stale), so two genuinely distinct
transactions never collide and an exact collision means a byte-identical copy --
precisely what must be refused.

Coinbase is exempt: a block may hold only one, its amount is pinned to the
height, and including someone else's would only donate the miner's own reward.

Three deliberate choices:

  * Enforced in Chain_AddBlockLocked, BEFORE the push. That function is the only
    insertion point into the chain besides the header-only disk load, so mining,
    broadcast, windowed sync, orphan attach and reorg apply are all covered by
    one check rather than four copies. Putting it in the ledger pass instead
    would be too late -- that runs after the block is in the chain and can only
    return false, leaving an invalid block behind.
  * Extracted as Chain_BlockRespectsSenderOrdering rather than left inline,
    so the multi-sender case can be tested directly. Senders are strictly
    independent: one account's timestamps say nothing about another's, and
    folding them together would reject ordinary blocks outright.
  * DebitAddress takes the timestamp and advances the guard in the same call, so
    a spend cannot happen without the guard moving. They cannot drift apart.

Rebuilt for free on reorg: the rollback already destroys and replays the whole
balance sheet, so setting lastTxTimestamp in that same loop means there is no
separate invalidation path to get wrong.

== The miner's fee sort broke the new rule ==

CompareTransactionPriority orders by fee descending, so a sender's later,
higher-fee transaction could sort ahead of their earlier, lower-fee one -- and
Chain_AddBlockLocked walks a block in order, so the node would have built blocks
its own rule rejects.

This cannot be folded into the comparator: "higher fee first, except same sender
by time" is not a strict weak ordering (A beats B on fee, B beats C on fee, C
beats A on time) and qsort with an inconsistent comparator is undefined. Instead
the priority sort is followed by a permutation restricted to each sender's own
slots, so fee-based slot allocation survives untouched and only the order within
one sender's slots changes.

== Mempool timestamp policy (local policy, NOT consensus) ==

Separate concern: keeping junk out of the pool. TX_MAX_FUTURE_DRIFT_MS (2h) and
TX_EXPIRY_MS (4 days, chosen to roughly match what DIFFICULTY_ADJUSTMENT_INTERVAL
spans but in milliseconds so it does not drift with block time), both in
constants.h. Gated at both admission sites via TxMempool_PolicyAccepts, with
TxMempool_PruneExpired on the 1Hz maintenance tick.

Blocks are never rejected for either bound, so a node with a skewed clock cannot
fork itself off the network over an admission rule.

Both bounds are measured against the node's own clock, NOT against the chain
tip. Measuring "future" against the last block assumes blocks keep arriving: on a
quiet chain the tip can be hours old, and an honest transaction created right now
would look hours ahead of it and be refused -- making sending impossible exactly
when the chain is idle.

A too-old timestamp needs no rule here; the replay guard already refuses anything
at or below a sender's last.

== Verification ==

New unit suite, 21 assertions against the real objects, all passing. The ones
that matter:

  multi-sender independence
    alice(6000) and bob(101) in one block          -> accepted
    reversed order                                  -> accepted
    8 senders sharing one timestamp                 -> accepted
  replay
    equal to sender's last (carbon copy)            -> rejected
    older than sender's last                        -> rejected
    same transaction twice in one block             -> rejected
  ordering within a block
    same sender increasing                          -> accepted
    same sender decreasing                          -> rejected
  policy window
    inside/outside both bounds, and pruning         -> as specified

Without the sender comparison in the guard, the first three would all fail --
that is the case worth guarding against, because the check reads as if it folds
all senders together.

penalty_test and dag_test suites still pass. AddressSanitizer reported zero
errors on both nodes across the reorg path, which is the signal that matters
given the rollback now does more work.

== Not yet verified ==

End-to-end replay rejection and reorg-restores-mempool against live nodes; both
need two funded wallets, so they are a separate setup. The fork regression
(forksync 5 60) was still climbing toward its penalty threshold when this was
written -- branchLead 40 of the required 42, behaving correctly but not yet
adopted.

== Note ==

balance_sheet_entry_t is written raw to disk, so balance_sheet.data gains a
field and old files will not load. wipechain before running.

Related gap, pre-existing and NOT addressed: the disk load restores headers only
and Chain_RecomputeRuntimeState does not rebuild balances, so lastTxTimestamp
survives a restart purely because it is persisted in balance_sheet.data -- and
that file has no height marker to detect it being stale against the chain. A node
with a missing or out-of-date sheet silently resets every account's guard to 0
and allows one replay per account. Balances are already wrong in that situation
today, but this change turns it from an accounting bug into a security one.
Recording the chain height alongside the sheet and refusing to start on a
mismatch is the natural follow-up.
 2026-08-03 16:50:16 +02:00
dcrubro 393d26dfcb Fix the difficulty adjustment off-by-one; slight initial diff adjustment; BREAKS CONSENSUS 2026-08-03 15:45:53 +02:00
dcrubro 5eaf0b699c Let a node that is behind adopt the peer's branch, and fix two bugs that only appear once a reorg actually applies 
Reported: a node at height N+10 on its own short fork, with the peer at N+500,
could never sync. It located the fork point correctly, pooled the branch, and
then refused it forever. At 90s blocks a node is 960 blocks behind after a day
offline, so this is the normal case, not an edge case.

== Why it could never recover ==

The reorg delay was served by LOCAL chain growth alone:

    elapsed = tipHeight - observedTip

That only makes sense for a node whose tip is advancing. A node that is behind
has a frozen tip precisely because it is rejecting the branch, so elapsed stays
0 forever while penalty(5) = 42. Both escape hatches also fail: mining out of it
means extending a fork nobody accepts, and the IBD exemption never fires for a
node that is mining, because mining keeps its tip fresh.

Being 490 blocks behind is being behind, not a reorg contest. Refusing to adopt
protects nothing while the node falls a further 40 blocks behind every hour.

The delay is now satisfiable by EITHER side making progress:

    localGrowth = tipHeight - observedTip
    branchLead  = candidateTip - tipHeight        (0 if not ahead)
    elapsed     = max(localGrowth, branchLead)

A branch already extending `penalty` blocks past our tip has demonstrated
exactly what the delay asks for, and every one of those blocks carries PoW we
validated ourselves. Requiring us to independently produce the same amount is
demanding the same proof twice. Only VALIDATED blocks count -- a peer's
advertised height is not evidence and never reaches this code.

It degrades correctly in both directions: a mining node's tip advances, so an
attacker must outpace it by penalty blocks; a node that is only observing
follows the heaviest chain, which is what an observer should do.

== The window has to keep pulling the branch ==

Every forked delivery reset nextReq back to our own tip, so the window
re-requested the same eight heights forever and the pool never grew past the
window size -- branchLead could not rise even in principle. The backward walk
now runs ONCE to establish linkage, then the window marches forward pooling the
branch, retrying adoption per window-full with a final attempt when it drains.

== sync force ==

Operator override that skips the delay for one sync, for a node whose chain is
known to be the wrong one. Threaded explicitly (Chain_ReplaceBranch gains
bypassPenalty, OrphanPool_AttemptAttachForced) rather than through a global, so
nothing a peer sends can reach it. Linkage, work comparison and atomicity still
apply -- it waives only the waiting, and says so loudly in the log.

== Bug found in testing: PoW is branch-relative ==

A block's epoch seed is the last block of the previous epoch ON ITS OWN BRANCH.
Validating a competing branch's block against OUR epoch seed does not merely
fail to resolve: when the chains diverge before the boundary it resolves to the
WRONG seed and rejects a perfectly valid block. Any fork spanning an epoch
boundary was therefore impossible to assemble -- the branch could not grow past
the boundary block, so branchLead stalled one short of it.

The receive path now does self-contained checks only (Block_HasValidStructure:
merkle, transactions, vote, non-empty). Proof of work moved to
Chain_AddBlockLocked, at the point a block joins the chain, where the branch
context is real -- the rollback has put its ancestors in place by then. That is
where the invariant belongs and it removes a duplicate check rather than adding
one. Needs Chain_DagParamsForHeightLocked, because Chain_AddBlockLocked already
holds chainLock for writing and the lock is not recursive.

Consequence worth knowing: the orphan pool can now hold blocks whose work has
not been verified, bounded by MAX_ORPHAN_BLOCKS (512). Each still had to pass
merkle and full transaction/signature validation, and none can reach the chain
unverified.

This bug also affected plain forward sync across block 350000; it was masked
because appending keeps the boundary block in the chain.

== Bug found in testing: stale DAG accepted as current ==

Block_PowHashHeavy matched on epoch index and size but not the seed. A DAG's
content is a function of (seed, size); the epoch index is a label for it. A
reorg is exactly the event that changes the seed while leaving index and size
untouched, so mid-apply the miner's context still held a DAG built from the
PRE-reorg seed, the guard passed, and a valid block was hashed against the wrong
lanes. g_dagSeed now records what each DAG was generated from and both
Block_EnsureAutolykos2Dag and Block_PowHashHeavy compare it.

== Bug found in testing: double free on the failed-apply path ==

SIGABRT in the allocator: free_tiny_botch -> DynArr_destroy -> Block_Destroy ->
Chain_FreeBlockArray -> Chain_ReplaceBranch.

DynArr_push_back stores the struct BY VALUE, so the chain's element and the
caller's block_t share one transactions pointer. Three places free that array
through the chain's copy -- Chain_ClearBlocks, Chain_RollbackToHeightLocked and
Chain_SaveToFile -- and each NULLs only the chain's side, leaving any caller
wrapper dangling. Whether a caller then had to use free() or Block_Destroy() was
a convention carried in comments at every call site plus a consumed-count passed
into Chain_FreeBlockArray. Chain_ReplaceBranch reset that count to 0 after
rolling back a failed apply, which told the cleanup to Block_Destroy exactly the
blocks whose arrays the rollback had just freed.

Rather than fix the count, the aliasing is now safe by construction:
Chain_AddBlockLocked clears the CALLER's transactions pointer immediately after
the push. Since DynArr_destroy(NULL) is a no-op, free(wrapper) and
Block_Destroy(wrapper) become equivalent and both safe regardless of what later
frees the chain's copy. The consumed-count parameter and both counters are gone
-- the thing that could be got wrong no longer exists -- and all call sites are
unified on Block_Destroy.

Placement is deliberate: immediately after the push, not at the end on success.
The ledger pass can fail with the block already in the chain, returning false to
a caller that destroys its wrapper on failure -- OrphanPool_ExtendTip does
exactly that, a third live instance not yet triggered.

Two follow-ons the refactor forced, both improvements anyway: MineAndAppendBlock
read the coinbase for its log line after the add (hoisted above it), and the
success log printed the caller's block rather than the chain's copy.

== Also ==

The deferral line is rate-limited. The maintenance thread retries pooled
branches once a second and elapsed only changes when something moves, so it
printed an identical line every second -- forever, on a node that is not mining.
It now reports each distinct situation once.

== Verification ==

Synthetic fork, node A 5 deep, node B ~60 ahead, EPOCH_LENGTH=8 so the branch
crosses three epoch boundaries:

  branchLead climbs 8 -> 16 -> 28 -> 34, crosses penalty(5)=42
  Adopted competing branch of 50 block(s) at fork height 20
  sync complete: localHeight=70
  Chain OK

Repeated under AddressSanitizer: adopted 47 blocks, 0 ASan errors on both nodes.
This matters because the refactor rewrites the exact cleanup path the SIGABRT
came from, and a double free that no longer aborts would otherwise pass silently.

Unit suites pass, including a new assertion "heavy path refuses a DAG built from
a different seed" -- the direct regression for the stale-DAG bug.

Harness note: each node needs its OWN wallet. With a shared one both pay the
same coinbase address, produce identical merkle roots, and at easy difficulty
mine byte-identical blocks -- the fork test silently became a catch-up test.

== Still untested ==

The restore-after-failed-apply path is no longer naturally reachable now that
the two bugs above are fixed, so it needs deliberate corruption to exercise.
Test B (branch only slightly ahead must still DEFER), test C (sync force), and a
TSan pass over the changed paths are outstanding.
 2026-08-02 19:36:09 +02:00
dcrubro 01c44731ef Fix windowed sync stalling against a peer on a competing branch 
Reported symptom: node A (4656 blocks, last 10 mined on its own fork) syncing
from node B (5646 blocks) crawled for minutes while B re-served the whole chain
several times over. A's log was thousands of "Queued orphan BLOCK_DATA" lines
interleaved with "timed out fetching block N, giving up", sliding forward 8
blocks at a time and never reorging.

== Root cause ==

The sync loop had no channel for FETCH_BLOCK replies. It inferred that a
requested block had arrived purely from the chain growing:

    if ((uint64_t)Chain_Size(chain) > h) { /* block h was applied */ }

But the reply is handled on the peer's io thread by Node_ParseAndAcceptBlock,
which correctly identifies a block whose prevHash points at the peer's branch
and files it in the orphan pool -- the chain does not grow. So a block that
ARRIVED AND WAS CORRECTLY ORPHANED was indistinguishable from a lost packet.

Every one of the 990 blocks therefore cost MAX_SYNC_RETRIES re-requests plus a
SYNC_REQUEST_TIMEOUT_MS stall before the window slid on and did it again. That
is both the crawl and the repeated serving on the peer.

Neither existing fork probe could rescue it:

  * The divergence check sits downstream of a SUCCESSFUL append -- it only runs
    after Chain_GetBlockCopy(chain, h) succeeds. When the fork begins at the
    very first requested height nothing ever lands, so RequestForkWindow was
    never reached from there.
  * The no-progress fallback only runs after the entire window range drains,
    i.e. after timing out through all 990 blocks, and is capped at 3 rounds.

== Delivery receipts ==

Node_NoteBlockDelivered / Node_TakeBlockDelivery / Node_ResetBlockDeliveries,
backed by a 512-slot ring with its own mutex. Recorded from the BLOCK_DATA
handler only -- BLOCK_DATA is sent solely in reply to a FETCH_BLOCK, so a
receipt means "the peer answered", independently of whether the block could
join our chain. A receipt for a height already present is refreshed in place,
so a retried request cannot leave a stale one for the next window to consume.

Receipts carry a status rather than a bool:

    NODE_DELIVERY_APPENDED   joined our chain
    NODE_DELIVERY_DUPLICATE  we already held this exact block -- common ground
    NODE_DELIVERY_ORPHANED   competing branch; now pooled
    NODE_DELIVERY_REJECTED   failed validation

DUPLICATE is what lets a backwards fork walk terminate; a plain "appended"
boolean cannot tell "we already have this" from "this is on their branch",
which is precisely the signal the walk needs.

The sync loop now treats any non-APPENDED delivery as the divergence signal and
probes immediately, instead of retrying something retrying cannot fix. Bounded
by MAX_FORK_PROBE_ROUNDS -- without that the window resets to the same height
and re-probes forever -- and the counter resets on real progress so a branch
that forks again further on can still be followed.

== RequestForkWindow is now an actual walk ==

It previously fired REORG_FETCH_DEPTH (128) requests and slept a flat
SYNC_REQUEST_TIMEOUT_MS. Two problems:

  * The sleep was a race against the peer's serving rate. At the ~10 blocks/s
    observed in testing, a 128-block window cannot land in 5s, so the
    OrphanPool_AttemptAttach that followed ran against a half-filled pool and
    reported a failure that was not real.
  * It always asked for the full depth. On a shallow fork the overwhelming
    majority came back as duplicates that were discarded without even entering
    the pool -- a wasted full block send each.

It now descends in batches of MAX_PARALLEL_FETCHES, waits on that batch's
receipts rather than guessing, and stops at the first height the peer returns
that we already hold: that block is the fork point and everything below it is
shared. It returns whether common ground was found, so callers no longer
attempt an attach that cannot possibly link.

Measured on the reported scenario: 16 requests instead of 129, locating the
fork point at height 4645 (fork = blocks 4646..4655, exactly the 10 mined).

== IBD_TIP_AGE_BLOCKS 500 -> 20 ==

The reorg penalty is served by LOCAL chain growth, so a node that does not mine
can never serve it -- its tip does not move, and the only way it could grow is
by adopting the branch the penalty is gating. The IBD exemption is that node's
only route back, so at 500 block times it had to sit stalled for ~12.5 hours.
20 block times is ~30 minutes at a 90s target, far beyond normal Poisson block
spacing (a gap that long has probability ~e^-20), so a node genuinely following
the tip will not trip it.

This is what flipped the live test from "initialSync=no" to "initialSync=yes"
and let the reorg be adopted at all. Noted in the constant's comment: the
exemption is all-or-nothing, so lowering it further widens that hole.

== Reorg failures are now legible ==

Chain_ReplaceBranch broke silently for fork-point-beyond-tip, work computation
failure, insufficient work, snapshot failure and candidate copy failure alike,
so "not adopted" could not be diagnosed from a log. Each now says which. The
not-heavier case reports candidate vs incumbent block counts, because the
common cause is a branch that is still arriving -- a partial branch is
genuinely lighter -- and that reads very differently from a peer actually on a
weaker chain.

The sync loop's "Reorg candidate not adopted (lighter branch, or still serving
its reorg penalty)" is replaced by "Reorg not completed on this pass; branch
stays pooled for retry". The old wording was actively misleading: live testing
showed it firing on reorgs that the 1Hz maintenance thread completed a second
later.

== Verification ==

Against a real peer at 5646 blocks, from a real local chain of 4656 with a
10-block fork:

                        reported    receipts only   all changes
  wall-clock            minutes     123s            116s
  timed-out fetches     1 per block 0               0
  fork-walk requests    n/a         129             16
  reorg completed       never       maintenance     first pass
  final height          4656        5646            5646
  fullverify            -           Chain OK        Chain OK

The 116s is not a meaningful speedup over 123s and should not be read as one:
990 blocks at ~10 blocks/s is ~100s, so both are bound by the peer's serving
throughput rather than by anything in the sync loop. The substantive wins are
the 129 -> 16 fork-walk reduction, the reorg completing deterministically
instead of by luck, and logs that no longer report failure on success.

Also exercised: a synthetic partitioned fork (10 vs 60 blocks) confirming zero
timeouts and bounded probe rounds when the branch is correctly refused by the
penalty.

== Known limits, unchanged ==

Chain_ReplaceBranch adopts whatever is pooled at attach time (18 blocks in the
live run); the remainder arrives through normal windowed sync afterwards. That
bounds a single reorg by MAX_ORPHAN_BLOCKS.
 2026-07-30 21:51:09 +02:00
dcrubro 0e721ca389 Fix reorg penalty scaling direction and rebuild DAG sizing around a miner-signalled band 
Two consensus-parameter bugs, both latent today and both guaranteed to become
live splits later. Neither is a regression; both are cheap to fix now and
expensive to fix after launch.

== 1. The reorg penalty scaled with block time the wrong way ==

FetchScheduler_ComputeReorgPenaltyBlocks had TARGET_BLOCK_TIME in the numerator
and REORG_PENALTY_REF_BLOCK_TIME in the denominator. The penalty is counted in
BLOCKS, so the wall-clock protection it actually buys was

    penalty(d) * TARGET_BLOCK_TIME  ~  d^2 * T^2 / REF

i.e. quadratic in block time, when the stated intent in constants.h is
wall-clock equivalence to the reference scheme defined at REF_BLOCK_TIME = 150.
At T = 90 the chain got 54*d^2 seconds of protection instead of 150*d^2 --
2.78x weaker than intended -- and any future block-time reduction would have
weakened it further, silently.

Swapping the two makes T cancel out of the wall-clock figure:

    penalty(d) = ceil(d^2 * REF / T)   ->   protection ~ d^2 * REF seconds

Penalty in blocks, depth -> before/after: 4 -> 10/27, 8 -> 39/107,
10 -> 60/167, 50 -> 1500/4167, 100 -> 6000/16667.

Verified by rebuilding at TARGET_BLOCK_TIME = 60: wall-clock protection is now
identical at both block times (2400s, 9600s, 15000s, ...), ratio 1.0000. Under
the old formula 60s blocks would have been 56% weaker.

Still integer-only: the ceiling division and saturation guards are unchanged,
and numeratorScale only moves 90 -> 150 while `raised` stays bounded by
REORG_PENALTY_MAX_DEPTH^2 = 1e6, nowhere near overflow.

This is fork choice, not block validity, so mixed-version nodes converge once
the longer penalty expires and no block ever becomes invalid.

== 2. DAG sizing was a fail-open consensus split waiting on block 350000 ==

Fixed, in rough order of severity:

  * FAIL-OPEN PoW. Block_CalculateAutolykos2Hash memset the hash to zero on any
    failure, and zero compares below every target -- so a DAG that was missing,
    mis-sized or had failed to build made *every* block pass PoW validation
    instead of rejecting it. PoW checks now fail closed.

  * TWO DIVERGENT IMPLEMENTATIONS of the same rule: CalculateTargetDAGSize in
    constants.h (mining) and ComputeEpochDagBytesForHeightFromChain in main.c
    (verification). They already disagreed at exactly height == EPOCH_LENGTH,
    where the constants.h copy underflowed size_t computing
    `Chain_Size - 1 - EPOCH_LENGTH` and returned 0.

  * A MEANINGLESS DIFFICULTY COUPLING. difficultyTarget is compact-encoded
    [1B exponent][3B coefficient]; subtracting two compact values mixes
    exponent and mantissa. The first real retarget yields a delta of 900649,
    which times DAG_BASE_GROWTH is ~967 TB, so the result was always clamped
    and the "proportional" term degenerated to a binary base+maxUp/base-maxDown
    switch. The size also never accumulated -- it was always recomputed from
    the constant base -- so the documented 1 GB/epoch growth could not happen.

  * A NON-EPOCH-ALIGNED SEED. GetNextDAGSeed returned hash(tip) while the
    verifier correctly used hash(block[epochIndex * EPOCH_LENGTH - 1]). The
    `chainSize % EPOCH_LENGTH == 0` rebuild guard masked this while running,
    but startup called it at an arbitrary height -- so a node restarted
    mid-epoch built a DAG from a different seed than one that had run straight
    through the boundary, and its blocks were rejected.

  * UNSIGNED PROMOTION. `DAG_BASE_GROWTH * difficultyDelta` promoted the signed
    delta to unsigned long long and wrapped before being assigned back to
    int64_t; the main.c copy cast first, so the two also differed in overflow
    behaviour. Both are gone.

  * A LEAK on the `targetSize <= 0` path in main.c (both block copies).

  * A USE-AFTER-FREE at every epoch boundary: Block_RebuildAutolykos2Dag ran
    DagClear -> DagAllocate -> DagGenerate from whichever thread advanced the
    tip, freeing ctx->dag.buf while miners read it.

  * GetAutolykos2Ctx called DagAllocate without DagGenerate, leaving
    dag.len == 0 so every heavy hash failed -- which, combined with the
    fail-open above, meant "everything is valid". It also memset 1 GiB that
    DagGenerate immediately overwrote.

  * TOCTOU / lock reentrancy: CalculateTargetDAGSize called Chain_Size 4x and
    Chain_GetBlockCopy 2x, each taking chainLock for reading, making it unsafe
    to call from a write-locked section.

--- New sizing rule: default-grow inside a hard band ---

DAG size now follows a recurrence gated by a miner signal in the block header,
clamped to [DAG_MIN_SIZE, DAG_MAX_SIZE]:

    brake  = (hold + down) * DAG_BRAKE_DEN > EPOCH_LENGTH * DAG_BRAKE_NUM
    downQ  =          down * DAG_DOWN_DEN  > EPOCH_LENGTH * DAG_DOWN_NUM

    downQ(k) && downQ(k-1) -> size -= DAG_EPOCH_STEP   (floored at DAG_MIN_SIZE)
    brake                  -> size unchanged
    otherwise              -> size += DAG_EPOCH_STEP   (capped at DAG_MAX_SIZE)

Growth is the default and there is deliberately NO up-vote: every signal a
miner can express only slows the walk or reverses it. Under stratum-style
pooled mining the pool builds the header and therefore controls its share of
the vote, so the mechanism has to be safe under pool capture -- and it is,
because the lever a pool would want (grow the DAG to price out smaller miners)
does not exist. This is NOT because upward capture would be self-defeating: it
would in fact be profitable, since the fixed block reward redistributes to
whoever survives and difficulty retargets down. The protection is the absence
of the lever. The whole upward trajectory is therefore governance
(DAG_EPOCH_STEP, DAG_MAX_SIZE), not signalling.

Braking keeps the DAG small, which helps old hardware and only costs ASIC
resistance -- bounded by DAG_MIN_SIZE, which is the constant that actually
secures the property. Shrinking needs a supermajority sustained across two
consecutive epochs; that gates the onset only, so miners genuinely being
squeezed get relief every epoch rather than every other one.

Thresholds are cross-multiplied rather than divided, so there is no rounding
for nodes to disagree on, and the denominator is the constant epoch length
rather than blocks-observed, so a partial epoch cannot read as a stronger
signal than it is. No floating point anywhere on this path.

--- Header vote field ---

reserved[0] carries the vote: 0 = grow (default), 1 = hold, 2 = down.
reserved[1..2] must be zero. All three already sat inside the packed, hashed
header, so the vote is committed to by both the canonical hash and the PoW hash
and cannot be altered after mining -- no wire-format or hash-layout change.

0 must mean grow, because the point of this shape is that inaction produces
growth; it also means a miner that knows nothing about the vote contributes to
the intended default rather than silently freezing the schedule.

Rejecting unrecognised vote values and non-zero spare bytes is a new validity
rule. It closes 24 bits of undefined-meaning malleable header space.

--- Validation moves to the light path ---

Autolykos2_DagGenerate fills lane i with exactly what ReadDagLaneFromSeed
recomputes for lane i -- Blake2b(seed || (i/2)_LE64), half i&1 -- so the DAG is
a pure cache and the two hashing paths are bit-for-bit equivalent. Validation
therefore uses the light path: no allocation, correct for any epoch rather than
only whichever one the global DAG happens to hold, and the DAG band becomes a
miner requirement rather than a full-node memory requirement.

Chain_OnTipAdvanced no longer rebuilds the DAG at all. MineBlock builds it on
demand for the height it is working on, so a node that does not mine never
allocates one, generation stays off the tip-advance path, and the buffer is
only ever touched by the miner (the ctx mutex remains as a backstop).

--- API changes ---

  + Chain_DagParamsForHeight(chain, height, &dagBytes, seed) -- single source
    of truth for both the size and the epoch seed, backed by a memoised
    per-epoch table on blockchain_t. The table is a pure cache of a function of
    the headers, extended lazily and dropped whenever anything at or below the
    tip changes; it lives on the chain rather than in a global because a
    second, header-only chain is built to re-verify historical PoW. Guarded by
    a per-chain mutex, always taken after chainLock.
  + Block_EnsureAutolykos2Dag / Block_PowHashHeavy / Block_PowHashLight.
    The heavy variant verifies the epoch and size itself, so it can never
    answer from a DAG built for another epoch.
  + Block_HasValidProofOfWorkWithParams -- resolve-once form. MineBlock called
    Block_HasValidProofOfWork inside its nonce loop, so resolving from the
    chain per attempt would have taken chainLock millions of times per block.
  + Block_HasValidVote.
  ~ Block_HasValidProofOfWork / Block_IsFullyValid now take the chain, because
    PoW validity genuinely is chain-relative. blockchain_t gained a struct tag
    so block.h can forward-declare it.
  - CalculateTargetDAGSize, GetNextDAGSeed, ComputeEpochDagBytesForHeightFromChain,
    ComputeEpochSeedForHeightFromChain, Block_CalculateAutolykos2Hash,
    Block_RebuildAutolykos2Dag, Autolykos2_LightHash (dead).
  - DAG_BASE_GROWTH and the five DAG_MAX_*_SWING_* / DAG_SWING_PERCENT_DEN
    macros, all now unreachable.

Also: Block_CalculateAutolykos2Hash truncated the height to uint32 while the
light path takes uint64, so the two would have diverged above block 2^32; the
full width is now passed. Added a `dagvote <grow|hold|down>` REPL command and a
progress line during DAG generation, which is tens of seconds at production
sizes.

static_asserts now enforce DAG_MIN_SIZE <= DAG_BASE_SIZE <= DAG_MAX_SIZE and
32-byte alignment, so a misconfigured band fails the build instead of being
silently clamped.

NOTE: DAG_MIN_SIZE (2 GiB), DAG_BASE_SIZE (2 GiB), DAG_MAX_SIZE (8 GiB) and
DAG_EPOCH_STEP (1 GiB) are economic judgements, not derivations, and since the
vote cannot accelerate growth they are the entire upward story. Sanity-check
them before launch.

== Verification ==

  * Penalty: exact table match at d = 4/8/10/50/100/1000, zero within grace,
    saturation at MAX_DEPTH, and identical wall-clock protection when rebuilt
    at TARGET_BLOCK_TIME = 60.
  * DAG recurrence (30 assertions against the real objects): default growth,
    legacy all-zero headers read as grow, strict inequality at exactly 1/2 and
    exactly 7/8, one qualifying epoch freezes but does not shrink, two
    consecutive shrink, sustained shrink repeats, both clamps saturate.
  * Epoch seed: constant across an epoch, equal to hash(block[k*EL - 1]),
    genesis seed in epoch 0, and resolvable at exactly height == EPOCH_LENGTH.
  * Fail-closed: unresolvable params and a zero-byte DAG both reject.
  * Heavy/light equivalence across 3 epochs; heavy refuses a DAG built for the
    wrong epoch or size.
  * Two nodes across 3 epoch boundaries: B reached height 30 purely by
    receiving, then mined blocks A accepted; both fullverify Chain OK, zero
    rejections.
  * Restart mid-epoch: B restarted at height 28 derived epoch 3's seed as
    hash(block[23]) -- the boundary block, not the tip -- and kept producing
    blocks A accepted. This fails before the change.
  * Divergent votes: A voting hold and B voting grow computed identical size
    and seed, confirming the tally is chain-derived, not config-derived.
  * ThreadSanitizer across epoch rollovers under load: no new races.
 2026-07-30 19:32:10 +02:00
dcrubro 4d39614cb5 Fix all ThreadSanitizer-reported races in the node lifecycle (8 reports -> 0) 
Found by running two nodes under -fsanitize=thread through connect -> mine -> broadcast -> clean
exit. None are in consensus code; all are connection setup/teardown.

Stop flags were plain or volatile ints written by one thread and read as a loop condition by
another. volatile stops the compiler hoisting the load but provides neither atomicity nor
ordering, which on arm64 is a real visibility gap, not just a sanitizer complaint. Now _Atomic:
 - net_node_t.maintenanceRunning  (Node_Destroy vs Node_MaintenanceThread)
 - tcp_server_t.isRunning         (TcpServer_Stop vs both accept threads) -- was a bare int, so
                                   the accept loop could legally be hoisted and never see the stop
 - udp_node_t.isRunning           (UdpNode_Stop vs the recv and retry threads)

TcpServer_Stop had a use-after-free, not merely a race: it took clientsMutex only long enough to
read the array pointer, then walked the slots unlocked. An exiting client thread clears its own
slot under that mutex and immediately destroys and frees the connection, so Stop could
RequestClose and pthread_join a freed pointer on any shutdown with an active peer.
 - Stop now requests the close and copies each pthread_t under the mutex, then joins from the
   copied handles, so the connection is never dereferenced outside the lock
 - the client thread's TcpConnection_Destroy/free moved inside the same critical section, which
   closes the window entirely

Client threads that disconnected normally were never joined and leaked their thread resources:
Stop only joins clients still present in the array, and a normal exit removes itself first. Stop
now claims each slot as it copies the handle, so the client thread can tell who owns its join --
it detaches itself if it successfully removed its own slot, and stays joinable if Stop already
claimed it. Both decisions happen under clientsMutex so the cases cannot interleave.

Node_Destroy cleared outbound slots with no lock (via TcpClient_Disconnect) while live inbound
client threads read the same field correctly under outboundLock in Node_HasLiveConnectionTo. The
lock cannot just be held across the destroy, because that path joins an io thread whose
on_disconnect callback takes outboundLock itself. Reworked to detach the connections from their
slots under the lock and tear them down outside it -- the pattern Node_ReapDeadOutbound already
uses in this file. This one only surfaced once the other five were fixed.

Verified: TSan clean over the same run; nodes still converge (height 25, identical tip); shallow
fork still adopts, depth-8 fork still defers on the reorg penalty, and the forced-orphan
regression still reaches full height with zero coinbase rejections.
 2026-07-28 23:07:57 +02:00
dcrubro 42b325d57a Fix self-deadlock on chainLock and fix double-delivery of accepted blocks (+ DUPLICATE enum entry) 2026-07-28 17:22:48 +02:00
dcrubro 1ff2890c0f Fix the reorg system: make rollback survivable, adopt by cumulative work, and enforce the reorg penalty everywhere 
(This is a big one, get ready - I told Claude to write the commit message cause I couldn't be bothered)

Root cause: reorg was broken at every layer and the failures compounded. Verified with the node's own
SKALACOIN_FORCE_ORPHAN_REORG debug mode, which stalled permanently at height 2 on 114 consecutive
coinbase-validation failures. A binary built at HEAD behaves identically, so none of this is a regression —
the reorg path had simply never worked.

Rollback (the keystone):
 - Chain_RollbackToHeight always returned false on any node that had ever saved or loaded its chain, and
   only after it had already truncated the chain and destroyed the balance sheet. Chain_RecomputeRuntimeState
   bails on any header-only block, and Chain_SaveToFile nulls transactions on every in-memory block once
   persisted, so the failure was universal in practice. Both callers treated the false as "nothing happened"
 - Added Chain_BorrowBlockTransactions / Chain_ReturnBlockTransactions, which fall back to the on-disk copy
   when the in-memory block has been compacted to headers
 - Supply is now accumulated in the rollback's existing balance-sheet replay pass instead of a second
   Chain_RecomputeRuntimeState pass
 - Gave Chain_RecomputeRuntimeState the same disk fallback: it had been failing on every restart with an
   existing chain, silently leaving currentSupply/currentReward at whatever came out of chain.meta

Fork choice is now cumulative work, not height:
 - Added Chain_ComputeBlockWork / Chain_ComputeWorkRange / Chain_ComputeBranchWork, computing
   2^256 / (target + 1) per block and summing over a range. Derived on demand from headers — no header,
   chain.meta or wire-format change
 - uint256 only had add/sub/cmp, so added uint256_divide (restoring binary long division),
   uint256_from_be_bytes, uint256_bitwise_not and uint256_is_zero
 - Comparison is strictly greater, so tied tips do not cause two nodes to keep swapping
 - Height-based choice was wrong now that difficulty actually varies: a long low-difficulty branch beat a
   short high-difficulty one

Atomic branch replacement:
 - Added Chain_ReplaceBranch: validate linkage, apply the reorg penalty, compare work, snapshot the outgoing
   blocks, roll back, apply. On any failure the original chain, balance sheet, supply, reward and difficulty
   target are restored. The caller keeps ownership of its blocks in every case — the chain applies copies
 - Split Chain_AddBlock and Chain_RollbackToHeight into locked public wrappers over unlocked internals, so a
   whole branch swap happens under one lock acquisition and Chain_OnTipAdvanced runs once per reorg rather
   than once per block
 - Chain_AddBlock now validates header.prevHash against the tip. It never did — that check lived only in
   Chain_IsValid and the network path, which is exactly why a rollback-then-reapply could splice blocks from
   two different forks into a chain that no longer links up
 - Moved the currentSupply/currentReward update into Chain_AddBlock. Each caller used to do it separately, so
   the orphan-attach and maintenance-thread paths never did, and the next block's coinbase was then validated
   against a stale currentReward and rejected forever. This was the height-2 stall

Reorg penalty (Horizen-style delayed block submission):
 - The penalty was only ever reachable from the manual sync command. The P2P broadcast -> orphan pool ->
   branch adoption path, which is the path an attacker actually uses, had none at all and picked the winner
   by raw height. It is now enforced inside Chain_ReplaceBranch, the single choke point every adoption
   passes through
 - Removed its application to the sync fetch window. The height gap to a peer is not a reorg depth;
   penalising it only throttled honest catch-up, and for gaps of 4-50 it collapsed the window to one block
   per pass, defeating MAX_PARALLEL_FETCHES
 - The depth is stamped once when a branch is first observed (orphan_entry_t.observedAtTipHeight) and never
   recomputed. Re-deriving it from a moving tip never converges: depth and elapsed both grow by one per block
   while penalty(depth) grows faster, so a penalized branch could never be adopted at all
 - The initial-sync exemption now comes from Chain_IsInitialBlockDownload, which uses the local median time
   past over MEDIAN_TIME_SPAN blocks. It used to key off the peer's advertised height, so any peer claiming
   localHeight + INITIAL_SYNC_HEIGHT_DIFF could switch reorg handling off for the whole session. A median
   rather than the tip alone means one backdated block cannot fake it either

Orphan pool (largely rewritten):
 - Added a pool mutex. It had no synchronisation whatsoever while being mutated from the 1 Hz maintenance
   thread, every per-peer TCP thread and the REPL thread; a concurrent insert could realloc the array while a
   scan held a raw element pointer. The lock is never held across a call into chain.c
 - Dedup by block hash, a MAX_ORPHAN_BLOCKS cap with oldest-first eviction, and pruning of entries that can
   no longer apply. Nothing was ever reaped before, and orphans are reachable before the chain-derived
   difficulty check, so this is also the memory-exhaustion fix
 - Candidate branches are now assembled by following prevHash from the fork point. Taking the first orphan
   found at each successive height could interleave blocks from two competing forks into one incoherent branch
 - Fixed rollbackHeight = forkHeight - 1. Chain_RollbackToHeight is exclusive, so every non-genesis adoption
   amputated one block too many and then failed Chain_AddBlock's index check
 - Fixed a block_t wrapper leak on every successful attach (free the wrapper, not Block_Destroy — the chain
   owns the transactions after a shallow copy)
 - Permanently invalid orphans are dropped instead of being retried on every maintenance tick forever

Forks below the tip are now discoverable:
 - A block at blockNumber < chainSize was rejected and freed, so the fork point and the lower half of any
   competing branch were always thrown away and a sub-tip fork could never be learned. Now the hash is
   compared: identical means a duplicate and is ignored, different means it goes to the orphan pool
 - The sync loop probes downwards (RequestForkWindow, bounded by REORG_FETCH_DEPTH and MAX_FORK_PROBE_ROUNDS)
   when it makes no progress while the peer is ahead. That is the only trigger that fires for a genuine
   sub-tip fork, because the old divergence check could only see blocks that had already entered our chain.
   FETCH_BLOCK already answers from the peer's own chain, so no protocol change was needed
 - Removed the rollback-to-height-0 path. "Could not find the parent" used to wipe the entire local chain,
   genesis included, and any peer could trigger it with a single unlinked block

Floating point removed from consensus and network math:
 - Chain_ComputeTargetAtHeight (the difficulty retarget) used double ratio arithmetic, and
   FetchScheduler_ComputeReorgPenaltyBlocks used double/pow/ceil. Both are consensus-critical and are now
   integer only; float results are not reproducible across platforms and compilers, and a single last-digit
   difference in a target or a penalty splits the network
 - The penalty constants became integer rationals (REORG_PENALTY_FACTOR_NUM/DEN, integer EXPONENT and
   REF_BLOCK_TIME) with saturating exponentiation and explicit ceiling division. Output is unchanged:
   penalty(4)=10, penalty(8)=39, penalty(10)=60, penalty(50)=1500, penalty(100)=6000
 - Removed the unused float macros DAG_MAX_UP/DOWN_SWING_PERCENTAGE and the now-dead math.h include from
   constants.h. Both were latent: DAG size feeds PoW verification. Replaced with integer numerator/denominator
   constants and used them at both clamp sites (values verified identical)

Other fixes that were blocking fork propagation:
 - madeProgressOverall was set but never reset, so after one productive pass the "no progress -> stop" guard
   could never fire again and the sync loop could spin forever holding the REPL
 - seenBlocks was inserted before/regardless of a successful send, so a block broadcast while no peer was
   connected was never offered again. It is now recorded only after the block actually goes out
 - Broadcasts relayed to outbound connections only, so in a two-node setup the dialled node never pushed
   anything back and the dialer learned of new blocks only via a manual sync. Inbound peers are now relayed to

Verified with two-node harnesses at a shortened adjustment interval:
 - forced-orphan regression: was height 2 with 114 coinbase rejections, now reaches the peer's height with
   zero rejections and both nodes report Chain OK
 - sub-tip fork at depth 3: the node discovers the fork below its own tip, discards its three blocks, adopts
   the heavier five, and both nodes converge on an identical tip hash
 - deep fork at depth 8: the strictly heavier branch is correctly refused with depth=8 penalty=39 elapsed=0
 - uint256 work arithmetic covered by a standalone test (division, big-endian conversion, monotonicity,
   halved target doubles work)

One issue found along the way: the chain in build/chain_data does not pass fullverify. Block 7683 reverts to
INITIAL_DIFFICULTY where it should carry 0x1f06df14, i.e. it contains blocks mined before 1288a64 landed. A
binary built at HEAD rejects it too, so this is stale data rather than a regression — it needs a wipechain
and a re-mine.
 2026-07-28 00:24:45 +02:00
dcrubro 1288a64977 Fixed the target computation to trigger on syncs - old system would cause difficulty mismatches on syncs, reorgs, etc. 2026-07-27 19:21:39 +02:00
dcrubro c16b88fc5a Add timestamp field to transactions - allows multiple transactions with the same Ins/Outs 2026-07-27 14:34:30 +02:00
dcrubro f9c785b316 Add timestamp field to transactions - allows multiple transactions with the same Ins/Outs 2026-07-27 14:30:06 +02:00
dcrubro 0e90f7d5db Fix multi-homed / IPv6 peer identity: nodes adding themselves and reconnect churn 
Root cause: peers were keyed only by (IP, listen port). An IPv6 host holds several addresses at once, so one node appeared as several peers and could not recognise its own addresses.

Protocol:
 - Added a random per-run node identity (localNodeId), advertised as a length-guarded trailing field in HELLO and ACK_HELLO — backwards compatible with peers that omit it
 - Added peerNodeId to tcp_connection_t; peers are now identified by this rather than by an endpoint
 - ACK_HELLO is now sent before any decision to drop the connection, so a rejected dialer learns whose address it reached instead of retrying forever

Self-connection:
 - Identity match → close the connection and record that endpoint permanently as our own
 - Self-endpoint set preemptively seeded from getifaddrs() at startup, so a node knows its own addresses before ever dialing one
 - Node_ConnectPeer refuses self endpoints, which also kills the echo-back chain that could exhaust connection slots

Duplicate connections and churn:
 - Dedup by identity per direction — one inbound and one outbound per physical peer, regardless of how many addresses it has
 - Moved dial history out of the peer table, so striking a peer no longer resets its connect-retry cooldown (this was the loop engine: strike → re-learn via gossip → redial next tick)
 - Node_HasOtherInboundFrom / Node_HasLiveConnectionTo now ignore connections that are already tearing down, and match on identity as well as endpoint

Gossip hygiene:
 - Never hand a peer its own other addresses in a PEERS reply (matched on identity, not just the socket address)
 - Reject unusable endpoints: link-local without scope id, unspecified, multicast, site-local (loopback stays allowed for local testing)
 - Normalise IPv4-mapped IPv6 so one host cannot occupy two entries

Two bugs found along the way:
 - All nodes drew the same identity — random_eight_byte() comes from srand(time(NULL)), so processes started in the same second produced identical values. Added random_secure_eight_byte() (/dev/urandom) for the identity, and mixed the pid into the seed so connection IDs stop colliding too
 - Identity dedup initially left inbound-only nodes mute — broadcasts traverse outbound connections only, so suppressing a dial-back because an inbound existed would have silenced such a node. Corrected to per-direction

Other:
 - peers output now shows each entry's node identity and the node's own endpoints
 - Added _DEFAULT_SOURCE to the build so getifaddrs() stays visible on glibc
 2026-07-27 14:08:57 +02:00
dcrubro 5aa99ecb01 Change functions IsValidIPv4/v6 to use addrinfo instead of custom logic 2026-07-27 12:31:03 +02:00
dcrubro 3e5c051645 Change functions IsValidIPv4/v6 to use addrinfo instead of custom logic 2026-07-27 12:28:34 +02:00
dcrubro c4e28df46f Fix txpooldetail command not working, add missing commands to list, fix connect command for ipv6 2026-07-27 12:22:43 +02:00
dcrubro 88a9caa46c Reap dead outbound connection slots; strike disconnected peers from discovery. Two related connection-lifecycle fixes on top of the SIGPIPE fix. 
Reclaim outbound slots on peer disconnect:
- TcpClient_ThreadProc fired on_disconnect but never cleared the outbound
  slot, closed the fd, joined the io thread, or freed the connection, so a
  dead peer permanently held its outboundClients[] slot. After MAX_CONS (32)
  churned connections the node could make no new outbound connections, and
  leaked fds/threads/memory. (The inbound side already self-reclaimed.)
- Add a reaper (Node_ReapDeadOutbound) on the maintenance thread: under
  outboundLock it detaches dead (disconnect-notified) slots, then joins the
  io thread and destroys/frees each connection outside the lock.
- Guard against use-after-free with a pin count on tcp_connection_t
  (TcpConnection_Pin/Unpin). The only cross-thread consumer holding a raw
  connection pointer across a blocking op is the `sync` command (via
  Node_GetBestOutboundPeer); it now pins the peer and unpins when done, and
  the reaper skips pinned connections. Discovery's snapshots run on the
  reaper's own thread, so they need no pin.
- Node_GetBestOutboundPeer/GetClientList/GetPeerEndpoints skip
  disconnect-notified connections so a dead peer is never handed out.
- Node_Destroy stops+joins the maintenance thread before tearing down
  outbound clients, so the reaper can't race shutdown.

Strike disconnected peers from the discovery peer list:
- Add NodeDiscovery_RemovePeer + Node_HandlePeerDisconnect: on disconnect,
  remove the peer from the known-peer table once no live connection (inbound
  or outbound) to its listen endpoint remains (Node_HasLiveConnectionTo;
  disconnect-notified conns don't count, so both directions dropping at once
  is handled). Wired into Node_Server_OnDisconnect and Node_Client_OnDisconnect.
 2026-07-26 23:08:21 +02:00
33 changed files with 4434 additions and 819 deletions
Expand all files Collapse all files
CMakeLists.txt
+377 -6
View File
@@ -9,6 +9,331 @@ set(CMAKE_C_STANDARD 23)
set(CMAKE_C_STANDARD_REQUIRED ON)
set(CMAKE_C_EXTENSIONS OFF)
# ---------------------------------------------------------
# Build configuration
#
# cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug # -Og -g3, ASan + UBSan
# cmake -S . -B build-strict -DCMAKE_BUILD_TYPE=Strict # Debug plus -Werror
# cmake -S . -B build-analyzer -DCMAKE_BUILD_TYPE=Analyzer # -Og -g3, GCC -fanalyzer
# cmake -S . -B build-release -DCMAKE_BUILD_TYPE=Release # -O3, _FORTIFY_SOURCE
#
# Debug finds memory bugs at run time, Analyzer finds them at compile time,
# Release survives them. Strict is Debug with warnings promoted to errors — the
# configuration CI should gate on, kept separate so a new warning never blocks
# someone mid-debugging. The warning set is identical in all four; only the
# instrumentation and the error policy differ.
#
# Debug and Analyzer are separate configurations rather than one Debug build
# with everything switched on, because the two tools actively interfere: ASan's
# instrumentation inflates the CFG enough that -fanalyzer exhausts its
# exploration budget and silently stops reporting. Measured on
# src/tcpd/tcpserver.c with GCC 16 — -fanalyzer alone reports the fd leak in
# TcpServer_Init, -fanalyzer plus ASan reports nothing at all.
#
# ThreadSanitizer is deliberately absent: it cannot be combined with ASan, so it
# would need a third configuration of its own.
# ---------------------------------------------------------
set(SKALACOIN_CUSTOM_CONFIGS Strict Analyzer)
set(SKALACOIN_BUILD_TYPES Debug Strict Analyzer Release RelWithDebInfo MinSizeRel)
get_property(SKALACOIN_MULTI_CONFIG GLOBAL PROPERTY GENERATOR_IS_MULTI_CONFIG)
if(SKALACOIN_MULTI_CONFIG)
foreach(_cfg IN LISTS SKALACOIN_CUSTOM_CONFIGS)
if(NOT "${_cfg}" IN_LIST CMAKE_CONFIGURATION_TYPES)
list(APPEND CMAKE_CONFIGURATION_TYPES ${_cfg})
endif()
endforeach()
elseif(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Debug CACHE STRING "Build type" FORCE)
message(STATUS "No CMAKE_BUILD_TYPE specified; defaulting to Debug")
endif()
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS ${SKALACOIN_BUILD_TYPES})
# Strict and Analyzer are custom configurations, so CMake has no built-in flags
# for them. Both inherit Debug's, and imported targets (OpenSSL::Crypto,
# CURL::libcurl) need a mapping because they only ship Debug/Release/NOCONFIG.
foreach(_cfg IN LISTS SKALACOIN_CUSTOM_CONFIGS)
string(TOUPPER ${_cfg} _cfg_upper)
foreach(_lang C CXX)
set(CMAKE_${_lang}_FLAGS_${_cfg_upper} "${CMAKE_${_lang}_FLAGS_DEBUG}"
CACHE STRING "Flags used by the ${_lang} compiler during ${_cfg} builds.")
mark_as_advanced(CMAKE_${_lang}_FLAGS_${_cfg_upper})
endforeach()
foreach(_linker EXE SHARED MODULE STATIC)
set(CMAKE_${_linker}_LINKER_FLAGS_${_cfg_upper} "${CMAKE_${_linker}_LINKER_FLAGS_DEBUG}"
CACHE STRING "Flags used by the linker during ${_cfg} builds.")
mark_as_advanced(CMAKE_${_linker}_LINKER_FLAGS_${_cfg_upper})
endforeach()
set(CMAKE_MAP_IMPORTED_CONFIG_${_cfg_upper} Debug "" Release RelWithDebInfo)
endforeach()
# Debug, Strict and Analyzer share the developer flag set (-Og -g3, warnings).
set(SKALACOIN_IS_DEBUGLIKE "$<OR:$<CONFIG:Debug>,$<CONFIG:Strict>,$<CONFIG:Analyzer>>")
# Debug and Strict are the runtime-instrumented pair; Analyzer must stay clean
# of sanitizers or -fanalyzer goes quiet (see above).
set(SKALACOIN_IS_SANITIZED "$<OR:$<CONFIG:Debug>,$<CONFIG:Strict>>")
# The Strict configuration always errors on warnings; this option additionally
# promotes them in Debug and Analyzer. OFF until src/ is clean under the warning
# set below — until then, build Strict when you want the gate.
option(SKALACOIN_WERROR "Debug/Analyzer: treat warnings as errors (always on in Strict)" OFF)
option(SKALACOIN_ENABLE_SANITIZERS "Debug config: AddressSanitizer + UndefinedBehaviorSanitizer" ON)
option(SKALACOIN_ENABLE_ANALYZER "Analyzer config: the compiler's static analyzer (GCC -fanalyzer)" ON)
# How much of the control-flow path -fanalyzer prints per report. 1 lists only
# the state transitions (opened here / first close here / leaks here), which is
# what you want while triaging; raise it when a report needs the branch-by-branch
# path that explains how it got there. GCC silently accepts out-of-range values,
# so validate here instead.
set(ANALYZER_VERBOSITY 1 CACHE STRING "Analyzer config: -fanalyzer path detail, 0 (terse) to 5 (full)")
set_property(CACHE ANALYZER_VERBOSITY PROPERTY STRINGS 0 1 2 3 4 5)
if(NOT ANALYZER_VERBOSITY MATCHES "^[0-5]$")
message(FATAL_ERROR "ANALYZER_VERBOSITY must be an integer from 0 to 5, got '${ANALYZER_VERBOSITY}'")
endif()
option(SKALACOIN_ENABLE_HARDENING "All configs: stack protector, _FORTIFY_SOURCE, RELRO/NOW, CFI" ON)
option(SKALACOIN_ENABLE_LTO "Optimized configs: link-time optimization" OFF)
include(CheckCCompilerFlag)
include(CheckLinkerFlag)
# Most of the interesting memory diagnostics below only exist from a certain
# GCC/Clang version onwards, and several are architecture specific. Probe every
# flag instead of gating on compiler version, so an older or foreign toolchain
# silently gets the subset it understands rather than failing to configure.
#
# A flag the driver merely tolerates is not a flag that does anything (Clang
# accepts -fstack-clash-protection on arm64 and then ignores it), so the probe
# promotes "argument unused" to an error where the compiler supports that.
check_c_compiler_flag(-Werror=unused-command-line-argument SKALACOIN_HAS_WERROR_UNUSED_ARG)
if(SKALACOIN_HAS_WERROR_UNUSED_ARG)
set(SKALACOIN_FLAG_PROBE_STRICT "-Werror=unused-command-line-argument")
else()
set(SKALACOIN_FLAG_PROBE_STRICT "")
endif()
function(skalacoin_append_supported_c_flags out_var)
set(_accepted ${${out_var}})
set(CMAKE_REQUIRED_FLAGS "${SKALACOIN_FLAG_PROBE_STRICT}")
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_CFLAG_${_flag}" _cache_var)
check_c_compiler_flag("${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
# Instrumentation such as -fsanitize=address has to reach the linker as well,
# and is only usable if the matching runtime library actually exists, so the
# probe hands the flag to both halves of the try-compile.
#
# Flags are also probed cumulatively, in the order given: some are only legal in
# the presence of an earlier one (-fsanitize=pointer-compare is rejected unless
# -fsanitize=address is already on the command line), and probing it alone would
# quietly drop it.
function(skalacoin_append_supported_instrument_flags out_var)
set(_accepted ${${out_var}})
set(_context "")
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_INSTRUMENT_${_flag}" _cache_var)
string(JOIN " " CMAKE_REQUIRED_FLAGS ${SKALACOIN_FLAG_PROBE_STRICT} ${_context})
set(CMAKE_REQUIRED_LINK_OPTIONS ${_context} "${_flag}")
check_c_compiler_flag("${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
list(APPEND _context "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
function(skalacoin_append_supported_link_flags out_var)
set(_accepted ${${out_var}})
foreach(_flag IN LISTS ARGN)
string(MAKE_C_IDENTIFIER "SKALACOIN_HAS_LDFLAG_${_flag}" _cache_var)
check_linker_flag(C "${_flag}" ${_cache_var})
if(${_cache_var})
list(APPEND _accepted "${_flag}")
endif()
endforeach()
set(${out_var} "${_accepted}" PARENT_SCOPE)
endfunction()
# Warnings for our own C code; the per-config flag sets are applied through
# generator expressions so multi-config generators (Xcode, VS) work too.
set(SKALACOIN_C_WARNINGS "")
set(SKALACOIN_C_FLAGS_DEBUGLIKE "")
set(SKALACOIN_C_FLAGS_OPTIMIZED "")
# The two instrumentation sets, each owned by one configuration and never both.
set(SKALACOIN_SANITIZER_FLAGS "")
set(SKALACOIN_ANALYZER_FLAGS "")
# Instrumentation that must be handed to the linker as well, and that also
# covers the vendored code we link in.
set(SKALACOIN_INSTRUMENT_COMPILE "")
set(SKALACOIN_INSTRUMENT_LINK "")
if(MSVC)
set(SKALACOIN_WERROR_FLAG /WX)
list(APPEND SKALACOIN_C_WARNINGS /W4 /permissive- /sdl)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE /Od /RTC1 /GS)
list(APPEND SKALACOIN_C_FLAGS_OPTIMIZED /O2 /GS /guard:cf)
if(SKALACOIN_WERROR)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE ${SKALACOIN_WERROR_FLAG})
endif()
if(SKALACOIN_ENABLE_ANALYZER)
list(APPEND SKALACOIN_ANALYZER_FLAGS /analyze)
endif()
if(SKALACOIN_ENABLE_SANITIZERS)
list(APPEND SKALACOIN_SANITIZER_FLAGS /fsanitize=address)
endif()
else()
set(SKALACOIN_WERROR_FLAG -Werror)
# Portable warning set, memory-safety first.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Wall
-Wextra
-Wpedantic
-Wshadow
-Wconversion
-Wsign-conversion
-Wcast-qual
-Wcast-align
-Wstrict-prototypes
-Wmissing-prototypes
-Wold-style-definition
-Wbad-function-cast
-Wwrite-strings
-Wformat=2
-Wnull-dereference
-Wdouble-promotion
-Wfloat-equal # consensus code must stay integer-only
-Wvla
-Wstack-protector
-Wpointer-arith
-Wundef
-Winit-self
-Wmissing-include-dirs
)
if(CMAKE_C_COMPILER_ID MATCHES "GNU")
# GCC-only diagnostics. The -W*=N forms ask for the strictest level:
# more false positives, but they catch out-of-bounds writes, dangling
# pointers and double frees that -Wall/-Wextra walk straight past.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Warray-bounds=2
-Wstringop-overflow=4
-Wstringop-truncation
-Wformat-overflow=2
-Wformat-truncation=2
-Wuse-after-free=3
-Wdangling-pointer=2
-Wfree-nonheap-object
-Walloc-zero
-Walloca
-Wduplicated-cond
-Wduplicated-branches
-Wlogical-op
-Wjump-misses-init
-Wtrampolines
-Wflex-array-member-not-at-end
)
# -fanalyzer is a whole-path symbolic execution pass (leaks, double
# free, use-after-free, NULL derefs across function boundaries).
if(SKALACOIN_ENABLE_ANALYZER)
skalacoin_append_supported_c_flags(SKALACOIN_ANALYZER_FLAGS
-fanalyzer-verbosity=${ANALYZER_VERBOSITY}
-fanalyzer
)
endif()
elseif(CMAKE_C_COMPILER_ID MATCHES "Clang")
# Clang-only diagnostics. Clang has no in-compiler equivalent of
# -fanalyzer, so the Analyzer config is simply an uninstrumented Debug
# build here — which is exactly the base `scan-build cmake --build
# build-analyzer` wants.
skalacoin_append_supported_c_flags(SKALACOIN_C_WARNINGS
-Warray-bounds-pointer-arithmetic
-Wconditional-uninitialized
-Wshift-sign-overflow
-Wassign-enum
-Wcomma
-Wloop-analysis
-Wthread-safety
-Wover-aligned
)
endif()
# -Og keeps the code steppable while still running the optimizer passes
# that -Wmaybe-uninitialized / -Wstringop-* rely on; at -O0 those warnings
# go quiet. -fno-omit-frame-pointer buys readable sanitizer backtraces.
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_DEBUGLIKE
-Og
-g3
-fno-omit-frame-pointer
)
if(SKALACOIN_WERROR)
list(APPEND SKALACOIN_C_FLAGS_DEBUGLIKE ${SKALACOIN_WERROR_FLAG})
endif()
if(SKALACOIN_ENABLE_HARDENING)
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_DEBUGLIKE
-fstack-protector-strong
)
# _FORTIFY_SOURCE needs an optimized build to see through the buffer
# sizes, and it fights ASan's interceptors, so it is Release-only.
skalacoin_append_supported_c_flags(SKALACOIN_C_FLAGS_OPTIMIZED
-U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=3
-fstack-protector-strong
-fstack-clash-protection
)
skalacoin_append_supported_instrument_flags(SKALACOIN_INSTRUMENT_COMPILE
-fcf-protection=full # x86_64 CET
-mbranch-protection=standard # aarch64 BTI/PAC
)
skalacoin_append_supported_link_flags(SKALACOIN_INSTRUMENT_LINK
"LINKER:-z,relro"
"LINKER:-z,now"
"LINKER:-z,noexecstack"
)
endif()
if(SKALACOIN_ENABLE_SANITIZERS)
# Everything that composes with ASan. LeakSanitizer is not listed
# because ASan already includes it where it is supported, and
# -fsanitize=leak cannot be combined with -fsanitize=address.
# -fno-sanitize-recover makes UB abort instead of printing and
# continuing, so a bad shift or overflow cannot be ignored in CI.
# pointer-compare/pointer-subtract additionally need
# ASAN_OPTIONS=detect_invalid_pointer_pairs=2 at run time.
skalacoin_append_supported_instrument_flags(SKALACOIN_SANITIZER_FLAGS
-fsanitize=address
-fsanitize-address-use-after-scope
-fsanitize=pointer-compare
-fsanitize=pointer-subtract
-fsanitize=undefined
-fsanitize=bounds-strict
-fno-sanitize-recover=undefined
-fno-omit-frame-pointer
)
endif()
endif()
# Each instrumentation set belongs to exactly one configuration. Keeping them in
# separate configs is the point of the Analyzer build, so never emit both.
if(SKALACOIN_SANITIZER_FLAGS)
list(APPEND SKALACOIN_INSTRUMENT_COMPILE "$<${SKALACOIN_IS_SANITIZED}:${SKALACOIN_SANITIZER_FLAGS}>")
list(APPEND SKALACOIN_INSTRUMENT_LINK "$<${SKALACOIN_IS_SANITIZED}:${SKALACOIN_SANITIZER_FLAGS}>")
endif()
if(SKALACOIN_ENABLE_LTO)
include(CheckIPOSupported)
check_ipo_supported(RESULT SKALACOIN_IPO_SUPPORTED OUTPUT SKALACOIN_IPO_ERROR)
if(NOT SKALACOIN_IPO_SUPPORTED)
message(WARNING "LTO requested but unsupported by this toolchain: ${SKALACOIN_IPO_ERROR}")
endif()
endif()
find_package(Threads REQUIRED)
include(FetchContent)
@@ -146,6 +471,11 @@ if(SKALACOIN_ENABLE_AUTOLYKOS2_REF)
add_library(autolykos2_ref STATIC ${AUTOLYKOS2_REF_SOURCES})
target_include_directories(autolykos2_ref PRIVATE ${AUTOLYKOS2_REF_BASE}/include)
# Vendored code gets the instrumentation but not our warning set: sanitizers
# only see a bug if the translation unit that owns the memory is compiled
# with them, and this library allocates buffers that our code touches.
target_compile_options(autolykos2_ref PRIVATE "${SKALACOIN_INSTRUMENT_COMPILE}")
target_link_options(autolykos2_ref PRIVATE "${SKALACOIN_INSTRUMENT_LINK}")
# Upstream source uses `malloc/free/exit/EXIT_FAILURE` without including
# stdlib headers in some C++ translation units. AppleClang can compile this,
# while Linux Clang fails. Force-include stdlib.h for C++ in this vendored lib.
@@ -186,7 +516,7 @@ set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
set(CMAKE_ARCHIVE_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/lib)
foreach(OUTPUTCONFIG DEBUG RELEASE RELWITHDEBINFO MINSIZEREL)
foreach(OUTPUTCONFIG DEBUG STRICT ANALYZER RELEASE RELWITHDEBINFO MINSIZEREL)
string(TOUPPER ${OUTPUTCONFIG} OUTPUTCONFIG_UPPER)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/bin)
set(CMAKE_LIBRARY_OUTPUT_DIRECTORY_${OUTPUTCONFIG_UPPER} ${CMAKE_BINARY_DIR}/lib)
@@ -220,15 +550,56 @@ endif()
target_include_directories(node PRIVATE
${PROJECT_SOURCE_DIR}/include
)
target_compile_options(node PRIVATE
-Wall
-Wextra
-Wpedantic
-g
# khash is vendored third-party code we cannot fix, and it accounts for half the
# warnings under Strict. SYSTEM turns -I into -isystem, which suppresses
# diagnostics from headers found through it. It needs its own search path: via
# ${PROJECT_SOURCE_DIR}/include the header resolves through the plain -I above
# and stays a normal header, so the sources include it as <khash.h>.
target_include_directories(node SYSTEM PRIVATE
${PROJECT_SOURCE_DIR}/include/khash
)
target_compile_options(node PRIVATE
"${SKALACOIN_C_WARNINGS}"
"${SKALACOIN_INSTRUMENT_COMPILE}"
"$<${SKALACOIN_IS_DEBUGLIKE}:${SKALACOIN_C_FLAGS_DEBUGLIKE}>"
"$<$<NOT:${SKALACOIN_IS_DEBUGLIKE}>:${SKALACOIN_C_FLAGS_OPTIMIZED}>"
# The static analyzer runs on our C sources only, never on the vendored
# C++ below, and never alongside the sanitizers.
"$<$<CONFIG:Analyzer>:${SKALACOIN_ANALYZER_FLAGS}>"
# Strict is Debug with the warning set turned into a build gate.
"$<$<CONFIG:Strict>:${SKALACOIN_WERROR_FLAG}>"
)
target_link_options(node PRIVATE "${SKALACOIN_INSTRUMENT_LINK}")
if(SKALACOIN_ENABLE_LTO AND SKALACOIN_IPO_SUPPORTED)
set_target_properties(node PROPERTIES
INTERPROCEDURAL_OPTIMIZATION_RELEASE ON
INTERPROCEDURAL_OPTIMIZATION_RELWITHDEBINFO ON
INTERPROCEDURAL_OPTIMIZATION_MINSIZEREL ON
)
endif()
target_compile_definitions(node PRIVATE
CHAIN_DATA_DIR="${CMAKE_BINARY_DIR}/chain_data"
$<$<BOOL:${SKALACOIN_AUTOLYKOS2_REF_AVAILABLE}>:SKALACOIN_AUTOLYKOS2_REF_AVAILABLE>
$<$<BOOL:1>:_POSIX_C_SOURCE=200809L>
# getifaddrs() (used to learn our own addresses) is a BSD extension, not POSIX; glibc hides it
# unless the default set is requested as well.
$<$<BOOL:1>:_DEFAULT_SOURCE>
)
set_target_properties(node PROPERTIES OUTPUT_NAME "skalacoin_node")
# ---------------------------------------------------------
# Configuration summary
# ---------------------------------------------------------
message(STATUS "skalacoin: build type ${CMAKE_BUILD_TYPE}")
message(STATUS "skalacoin: compiler ${CMAKE_C_COMPILER_ID} ${CMAKE_C_COMPILER_VERSION}")
message(STATUS "skalacoin: warnings-as-err ${SKALACOIN_WERROR} (always on in Strict)")
message(STATUS "skalacoin: sanitizers [Debug/Strict] ${SKALACOIN_SANITIZER_FLAGS}")
message(STATUS "skalacoin: static analyzer [Analyzer] ${SKALACOIN_ANALYZER_FLAGS}")
message(STATUS "skalacoin: hardening ${SKALACOIN_ENABLE_HARDENING}")
message(STATUS "skalacoin: LTO ${SKALACOIN_ENABLE_LTO}")
if(CMAKE_BUILD_TYPE STREQUAL "Analyzer" AND NOT SKALACOIN_ANALYZER_FLAGS)
message(STATUS "skalacoin: NOTE - Analyzer config has no static analyzer on "
"${CMAKE_C_COMPILER_ID}; use scan-build over this build tree")
endif()
TODO.txt
+3 -3
View File
@@ -14,9 +14,6 @@ Check if Block FullVerify is actually verifying fully (not missing any condition
A loophole in the reorg penalty system could potentially exist where someone broadcasts blocks one-at-a-time. Determine a solution to this.
IPv6 support for the P2P node. Come on guys, it's 2026. RFC 2460 was in 1998. It's about time.
Like if someone is behind NAT, fine, workable. CGNAT? Lmao good luck.
TO TEST:
Implement Horizen's "Reorg Penalty" system to make it harder for the young chain to be attacked by a powerful miner.
@@ -31,3 +28,6 @@ a constant inflation rate of 1.5% per year. It's lower than fiat (USD is ~2.8% p
Move to a GPU algo. RandomX is a good candidate, but CPU mining is not that attractive to anyone but people who actually want to support the project.
Sadly, CPUs won't incentivize people who want to profit, which let's be fair, is the majority of miners.
IPv6 support for the P2P node. Come on guys, it's 2026. RFC 2460 was in 1998. It's about time.
Like if someone is behind NAT, fine, workable. CGNAT? Lmao good luck.
include/autolykos2/autolykos2.h
+3 -1
View File
@@ -32,7 +32,9 @@ bool Autolykos2_Hash(
uint8_t outHash[32]
);
bool Autolykos2_LightHash(const uint8_t* seed, blockchain_t* chain, uint64_t nonce, uint8_t* out);
// Derives the DAG lanes it needs straight from the epoch seed, so it needs no DAG allocation and
// stays correct for any height regardless of which epoch a DAG happens to be built for. Produces
// exactly the same hash as Autolykos2_Hash against a DAG generated from the same seed and size.
bool Autolykos2_LightHashAtHeight(
const uint8_t seed32[32],
const uint8_t* message,
include/balance_sheet.h
+25 -1
View File
@@ -6,7 +6,7 @@
#include <stdbool.h>
#include <stdlib.h>
#include <stdio.h>
#include <khash/khash.h>
#include <khash.h>
#include <crypto/crypto.h>
#include <block/transaction.h>
#include <string.h>
@@ -16,10 +16,34 @@
typedef struct {
uint8_t address[32]; // For now just the SHA-256 of the public key; allows representation in different encodings (base58, bech32, etc) without changing the underlying data structure
uint256_t balance;
/**
* Timestamp (unix ms) of the most recent transaction this address SENT that is in the chain.
*
* Replay protection. Without it any historical transaction could be rebroadcast and mined a
* second time, debiting the sender again -- with UTXOs the spent inputs make that impossible,
* but an account model has nothing to stop it. A non-coinbase transaction is only valid if its
* timestamp is strictly greater than this, so a byte-identical replay (same timestamp, same
* hash) can never be included twice. Enforced in Chain_AddBlockLocked; see the note there.
*
* Rebuilt for free by the rollback's balance-sheet replay, so a reorg cannot leave it stale.
* Persisted with the rest of the entry -- note the file has no height marker, so a balance
* sheet that is out of sync with the chain silently resets this to 0 for every account.
**/
uint64_t lastTxTimestamp;
// TODO: Additional things
} balance_sheet_entry_t;
// KHASH_INIT expands to khash's own implementation, which is not -Wconversion
// clean. -isystem silences the header itself but not code expanded from its
// macros, because the diagnostic is attributed to this line.
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wconversion"
#endif
KHASH_INIT(balance_sheet_map_m, key32_t, balance_sheet_entry_t, 1, hash_key32, eq_key32)
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif
extern khash_t(balance_sheet_map_m)* sheetMap;
void BalanceSheet_Init();
include/block/block.h
+60 -5
View File
@@ -18,7 +18,10 @@ typedef struct {
uint8_t merkleRoot[32];
uint32_t difficultyTarget; // Encoding: [1 byte exponent][3 byte coefficient]; Target = coefficient * 256^(exponent-3)
uint8_t version;
uint8_t reserved[3]; // 3 bytes (Explicit padding for 8-byte alignment)
// reserved[0] carries the miner's DAG-size vote (DAG_VOTE_* in constants.h); reserved[1..2] must
// be zero. All three are inside the hashed header, so a vote is committed to by both the
// canonical hash and the PoW hash and cannot be altered after the block is mined.
uint8_t reserved[3];
} block_header_t;
#pragma pack(pop)
@@ -27,17 +30,69 @@ typedef struct {
DynArr* transactions; // Array of signed_transaction_t, NOTE: Potentially move to a hashmap at some point for quick lookups.
} block_t;
// PoW validity is chain-relative: it needs the epoch DAG size and seed. chain.h includes this
// header, so the tag declared there is forward-declared here to break the cycle.
typedef struct blockchain blockchain_t;
block_t* Block_Create();
void Block_CalculateHash(const block_t* block, uint8_t* outHash);
void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash);
void Block_CalculateAutolykos2Hash(const block_t* block, uint8_t* outHash);
bool Block_RebuildAutolykos2Dag(size_t dagBytes, const uint8_t seed32[32]);
void Block_AddTransaction(block_t* block, signed_transaction_t* tx);
void Block_RemoveTransaction(block_t* block, uint8_t* txHash);
bool Block_HasValidProofOfWork(const block_t* block);
/**
* Autolykos2 PoW hashing.
*
* The heavy variant reads its lanes from the process-global DAG and is a MINING accelerator only;
* the light variant derives the same lanes from the epoch seed on demand. They are bit-for-bit
* equivalent by construction -- Autolykos2_DagGenerate fills lane i with exactly what
* ReadDagLaneFromSeed recomputes for lane i -- so a block mined through either verifies through
* either. Validation always uses the light path: it needs no allocation, which is what keeps the
* DAG a miner requirement rather than a full-node memory requirement, and it stays correct for
* blocks from earlier epochs (the heavy path can only ever answer for whichever epoch the global
* DAG was last built for).
**/
bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8_t seed32[32]);
// Fails rather than answering from a DAG built for a different epoch, size OR SEED, so it can
// never silently hash against the wrong lanes. The seed matters because a reorg changes it while
// leaving the epoch index and size unchanged.
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]);
bool Block_PowHashLight(const block_t* block, size_t dagBytes, const uint8_t seed32[32], uint8_t outHash[32]);
// PoW check against explicitly supplied epoch parameters, for callers that resolve them once and
// then iterate (the miner). Returns false if the hash cannot be computed -- never treat an
// uncomputable proof as valid.
bool Block_HasValidProofOfWorkWithParams(const block_t* block, uint64_t epochIndex,
size_t dagBytes, const uint8_t seed32[32]);
// PoW check that resolves the epoch parameters for the block's own height from `chain`.
bool Block_HasValidProofOfWork(const block_t* block, blockchain_t* chain);
// Header vote field is a recognised value and the unused reserved bytes are zero.
bool Block_HasValidVote(const block_t* block);
bool Block_AllTransactionsValid(const block_t* block);
bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees);
bool Block_IsFullyValid(const block_t* block);
/**
* Self-contained validity: merkle root, transactions, vote encoding, non-empty. Needs no chain, so
* it is meaningful for ANY block, including one on a branch we do not have.
*
* This is what the receive path checks. Proof of work is deliberately NOT checked there, because
* PoW is only meaningful relative to the branch a block belongs to: the epoch seed is the last
* block of the previous epoch on ITS OWN branch. Validating a competing branch's block against our
* epoch seed does not merely fail to resolve -- when the two chains diverge before the boundary it
* resolves to the WRONG seed and rejects a perfectly valid block, which made any fork spanning an
* epoch boundary impossible to assemble.
*
* Chain_AddBlock verifies proof of work at the moment a block joins the chain, where the branch
* context is real. That, not the receive path, is what enforces the invariant.
**/
bool Block_HasValidStructure(const block_t* block);
// Full check including chain-relative PoW. Only meaningful for a block that extends `chain`.
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain);
void Block_ShutdownPowContext(void);
void Block_Destroy(block_t* block);
void Block_Print(const block_t* block);
include/block/chain.h
+117 -1
View File
@@ -7,13 +7,41 @@
#include <stdio.h>
#include <stdbool.h>
#include <string.h>
#include <pthread.h>
#include <uint256.h>
#include <storage/block_table.h>
#include <balance_sheet.h>
// One entry of the memoised DAG size recurrence, one per epoch. See Chain_DagParamsForHeight.
typedef struct {
uint64_t sizeBytes; // DAG size used by every block whose height falls in this epoch
bool downQualified; // this epoch's own votes met the down supermajority
} dag_epoch_state_t;
// Tagged so block.h can forward-declare it: PoW validity depends on the chain (it needs the epoch
// seed), but chain.h includes block.h, so the tag is what breaks the cycle.
typedef struct blockchain {
DynArr* blocks;
size_t size;
/**
* Memoised DAG size recurrence: a pure cache of a function of the block headers, extended
* lazily and dropped whenever anything at or below the tip changes (every epoch's size depends
* on the votes of every epoch before it). It lives on the chain rather than in a global because
* a second, header-only blockchain_t is built to re-verify historical PoW, and the two must not
* share a cache.
*
* `dagEpochsComputed` counts valid `sizeBytes` entries. `downQualified` is only filled in for
* an epoch once the *following* entry has been computed, so it is valid on
* [0, dagEpochsComputed - 1).
*
* Guarded by `dagCacheLock`, which is always taken AFTER `chainLock` and is never held across a
* call back into chain.c.
**/
dag_epoch_state_t* dagEpochs;
size_t dagEpochsComputed;
size_t dagEpochsCapacity;
pthread_mutex_t dagCacheLock;
} blockchain_t;
blockchain_t* Chain_Create();
@@ -28,6 +56,54 @@ void Chain_Wipe(blockchain_t* chain);
// Returns true on success.
bool Chain_RollbackToHeight(blockchain_t* chain, size_t height);
/**
* Atomically replace the blocks at [forkHeight, tip] with `newBlocks` (ascending, `count` of them).
*
* The swap happens only if the candidate branch is properly linked, has strictly more cumulative
* work, and has served its Horizen delayed-submission penalty. `observedAtTipHeight` is the local
* tip height at which the branch was FIRST seen and must not be recomputed as the chain grows --
* see the comment in the implementation. The initial-block-download exemption is decided inside,
* from local state only, so no caller can switch the penalty off.
*
* `bypassPenalty` skips the delay check ONLY. It exists for an explicit operator action (`sync
* force`) on a node whose chain is known to be the wrong one -- the penalty is served by local
* chain growth, so a node that is neither mining nor stale enough to count as catching up cannot
* clear it on its own. It must never be reachable from anything a peer says; work comparison,
* linkage and atomicity are still enforced, so this cannot adopt a branch that is not heavier.
*
* On any failure the original chain, balance sheet, supply and reward are restored and false is
* returned. The caller keeps ownership of `newBlocks` in every case: the chain applies copies.
**/
bool Chain_ReplaceBranch(blockchain_t* chain,
size_t forkHeight,
block_t** newBlocks,
size_t count,
uint64_t observedAtTipHeight,
bool bypassPenalty);
// True when this node is catching up rather than following the tip (empty chain, or a median
// block time far in the past). Used to exempt initial sync from the reorg penalty.
bool Chain_IsInitialBlockDownload(blockchain_t* chain);
// Penalty in blocks of local chain growth before a branch forking `reorgDepth` blocks back may be
// adopted. Thin wrapper over FetchScheduler_ComputeReorgPenaltyBlocks, for callers that only
// want to report it.
uint64_t Chain_ReorgPenaltyForDepth(uint64_t reorgDepth);
/**
* Replay guard: true if every non-coinbase transaction in `block` is newer than its own sender's
* last included transaction, and newer than that same sender's earlier transactions in this block.
*
* Reads the balance sheet's per-account `lastTxTimestamp` (see balance_sheet.h). Senders are
* considered independently -- one account's transactions say nothing about another's ordering, so
* an ordinary block full of different senders always passes. Coinbase is exempt.
*
* Exposed rather than inlined so this can be tested directly; Chain_AddBlockLocked calls it as part
* of block validation, which is what makes it apply to mining, sync, broadcast, orphan attach and
* reorg alike.
**/
bool Chain_BlockRespectsSenderOrdering(const block_t* block);
// Recompute `currentSupply` and `currentReward` from the in-memory chain blocks.
// Returns true on success and updates runtime state globals.
bool Chain_RecomputeRuntimeState(blockchain_t* chain);
@@ -41,6 +117,46 @@ bool Chain_LoadFromFile(blockchain_t* chain, const char* dirpath, uint256_t* out
bool Chain_LoadBlockFromFile(const char* dirpath, uint64_t blockNumber, bool loadTransactions, block_t** outBlock, size_t* outTxCount);
// Difficulty
uint32_t Chain_ComputeNextTarget(blockchain_t* chain, uint32_t currentTarget);
// Retarget for the block at `height`, measured over the window [height - INTERVAL, height - 1].
// `chain` must hold blocks 0..height-1. Takes no locks; safe to call while holding `chainLock`.
uint32_t Chain_ComputeTargetAtHeight(blockchain_t* chain, uint64_t height, uint32_t currentTarget);
// The consensus-required difficultyTarget for the block at `height`, derived from the chain alone.
// Takes no locks; safe to call while holding `chainLock`.
uint32_t Chain_GetTargetForHeight(blockchain_t* chain, uint64_t height);
// Refresh runtime state derived from the chain tip (difficulty target, epoch DAG).
// Call after any change to the tip. Must NOT be called while holding `chainLock`.
void Chain_OnTipAdvanced(blockchain_t* chain);
// DAG
/**
* The Autolykos2 DAG size and epoch seed that the block at `blockHeight` must be hashed against.
*
* This is the single source of truth for both, so the mining path and the verification path cannot
* drift apart. Size follows the default-grow recurrence gated by the miner votes in
* `header.reserved[0]` (see the DAG band in constants.h); the seed is epoch-aligned -- epoch 0 uses
* the genesis seed, epoch k uses the hash of the last block of epoch k-1 -- so it is constant for
* the whole epoch rather than changing every block.
*
* Requires the chain to hold every block below the start of `blockHeight`'s epoch, which is always
* true when validating or mining a block at that height. Returns false if it cannot produce both
* values; callers MUST treat that as an invalid proof rather than falling back to a default.
*
* Takes `chainLock` for reading internally. Must NOT be called while holding it.
**/
bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]);
// Work
// Expected number of hashes to satisfy `difficultyTargetBits`, i.e. 2^256 / (target + 1).
bool Chain_ComputeBlockWork(uint32_t difficultyTargetBits, uint256_t* outWork);
// Summed work of the chain's blocks over the half-open range [from, to).
// Takes no locks; safe to call while holding `chainLock`.
bool Chain_ComputeWorkRange(blockchain_t* chain, size_t from, size_t to, uint256_t* outWork);
// Summed work of a candidate branch that is not (yet) part of the chain.
bool Chain_ComputeBranchWork(block_t** blocks, size_t count, uint256_t* outWork);
#endif
include/block/transaction.h
+6 -1
View File
@@ -20,9 +20,14 @@ static inline bool Address_IsCoinbase(const uint8_t address[32]) {
return true;
}
// 160 bytes total for v1
// 168 bytes total for v1
#pragma pack(push, 1) // Ensure no padding for consistent file storage
typedef struct {
uint64_t timestamp; // Unix timestamp in MILLISECONDS (get_current_time_ms). Two of a sender's
// transactions must have strictly increasing timestamps -- see
// lastTxTimestamp in balance_sheet.h. Millisecond resolution is what makes
// an exact collision mean 'byte-identical replay' rather than 'two real
// transactions that happened to coincide'.
uint64_t fee; // Rewarded to the miner; can be zero, but the miner may choose to ignore transactions with very low fees
uint64_t amount1;
uint64_t amount2;
include/constants.h
+159 -99
View File
@@ -33,8 +33,26 @@
#define DIFFICULTY_ADJUSTMENT_INTERVAL 3840 // Every 3840 blocks (roughly every 4 days with a 90 second block time)
// Max adjustment per is x2. So if blocks are coming in too fast, the difficulty will at most double every 24 hours, and vice versa if they're coming in too slow.
#define TARGET_BLOCK_TIME 90 // Target block time in seconds
//#define INITIAL_DIFFICULTY 0x1f0c1422 // Default compact target used by Autolykos2 PoW (This is ridiculously low)
#define INITIAL_DIFFICULTY 0x1f1b7c51 // This takes 90s on my machine with a single thread, good for testing
// The retarget measures the span between the FIRST and LAST block of the window, which is one fewer
// interval than the window has blocks, and divides by it. Two blocks is the minimum that leaves a
// non-zero span. See Chain_ComputeTargetAtHeight.
static_assert(DIFFICULTY_ADJUSTMENT_INTERVAL >= 2,
"DIFFICULTY_ADJUSTMENT_INTERVAL must span at least one block interval");
#define INITIAL_DIFFICULTY 0x1f0c1422 // Default compact target used by Autolykos2 PoW (This is ridiculously low)
//#define INITIAL_DIFFICULTY 0x1f1b7c51 // Ridiculously low difficulty for testing.
// Mining
// The timestamp lives in the header the PoW hashes, so the miner restamps it while searching rather
// than keeping the one stamped when the search started. Two things fall out of that: a block carries
// the time it was actually found instead of a timestamp that is a whole block time stale on average,
// and every restamp is a fresh search space, so the nonce sweep starts over from 0 and never has to
// walk out to keep finding untried candidates. It costs nothing to throw the old nonce range away --
// each attempt is independent, so the work already done was never getting any closer.
static const uint64_t MINING_TIMESTAMP_REFRESH_MS = 2ULL; // Don't restamp for a drift smaller than this
// Reading the clock once per hash would be wasted work next to a memory-hard hash, so the check is
// batched. Note this, not the refresh interval, is what actually bounds accuracy once a batch of
// hashes takes longer than MINING_TIMESTAMP_REFRESH_MS -- keep it small enough that it doesn't.
static const uint64_t MINING_TIMESTAMP_CHECK_NONCES = 16ULL;
// Sync / Reorg tuning constants
// Timeouts and retry/backoff behavior for block fetches during sync (milliseconds)
@@ -43,14 +61,79 @@ static const int MAX_SYNC_RETRIES = 4; // retry attempts per block fetch
static const uint64_t SYNC_BACKOFF_BASE_MS = 200ULL; // base backoff in ms (exponential)
// Parallelism
static const int MAX_PARALLEL_FETCHES = 8; // concurrent block fetches during windowed sync
// Heuristic: if peer is this many blocks ahead, treat as initial sync
static const uint64_t INITIAL_SYNC_HEIGHT_DIFF = 50ULL;
// How far below a detected divergence we ask a peer for blocks, so the orphan pool has enough of
// the competing branch to locate the fork point by prevHash linkage.
static const uint64_t REORG_FETCH_DEPTH = 128ULL;
// How many times one `sync` will probe downwards for a fork point before giving up, so a peer on a
// permanently incompatible chain cannot keep us looping.
static const int MAX_FORK_PROBE_ROUNDS = 3;
// Reorg penalty configuration (used to penalize peers reporting higher heights but with delayed work)
// Reorg penalty configuration (Horizen-style delayed block submission penalty).
// A branch forking B blocks below our tip is held for penalty(B) blocks of local chain growth
// before it may be adopted, so a rented-hashrate attacker has to sustain the attack publicly
// instead of winning by dumping a privately mined branch.
//
// penalty(B) = ceil(FACTOR_NUM/FACTOR_DEN * B^EXPONENT * REF_BLOCK_TIME / TARGET_BLOCK_TIME)
//
// The block-time ratio is REF/TARGET, not TARGET/REF. penalty() counts BLOCKS, so the wall-clock
// protection is penalty(B) * TARGET_BLOCK_TIME ~= B^EXPONENT * REF_BLOCK_TIME: TARGET_BLOCK_TIME
// cancels and the protection is block-time-independent. See fetch_scheduler.c.
//
// Expressed as integer rationals on purpose: this feeds fork choice, so it must evaluate
// identically on every node. Floating point is not acceptable here.
static const uint64_t REORG_PENALTY_GRACE_BLOCKS = 3ULL; // allow small reorgs without penalty
static const double REORG_PENALTY_FACTOR = 1.0; // base scaling factor (theta)
static const double REORG_PENALTY_EXPONENT = 2.0; // exponent p in penalty ~ B^p
static const double REORG_PENALTY_REF_BLOCK_TIME = 150.0; // reference block time in seconds used by original scheme
static const uint64_t REORG_PENALTY_FACTOR_NUM = 1ULL; // base scaling factor (theta), numerator
static const uint64_t REORG_PENALTY_FACTOR_DEN = 1ULL; // base scaling factor (theta), denominator
static const uint32_t REORG_PENALTY_EXPONENT = 2U; // exponent p in penalty ~ B^p
static const uint64_t REORG_PENALTY_REF_BLOCK_TIME = 150ULL; // reference block time in seconds used by original scheme
// Beyond this depth the penalty saturates. At the configured parameters penalty(1000) is already
// ~1.67M blocks (~4.75 years at a 90s block time), so this only exists to keep the arithmetic away
// from overflow rather than to bound the penalty in any meaningful sense.
static const uint64_t REORG_PENALTY_MAX_DEPTH = 1000ULL;
/**
* Mempool transaction timestamp policy. LOCAL POLICY, NOT CONSENSUS.
*
* These govern what this node is willing to hold and relay; a block containing a transaction that
* violates either is still accepted. That separation is deliberate -- a node with a skewed clock
* must not be able to fork itself off the network over an admission rule.
*
* A too-OLD timestamp needs no rule here: the per-account replay guard (see balance_sheet.h) already
* refuses anything at or below a sender's last included transaction.
**/
// Refuse to admit a transaction dated further ahead than this of OUR OWN CLOCK. Measured against
// the clock and not against the chain tip on purpose: on a quiet chain the tip can be hours old, and
// judging "future" against it would refuse honest transactions exactly when blocks are sparse.
static const uint64_t TX_MAX_FUTURE_DRIFT_MS = 2ULL * 60ULL * 60ULL * 1000ULL; // 2 hours
// Drop transactions older than this from the mempool, so it is not inflated by junk that will never
// be mined. Roughly the ~4 days DIFFICULTY_ADJUSTMENT_INTERVAL spans, but expressed in milliseconds
// so it does not drift if the block time changes.
static const uint64_t TX_EXPIRY_MS = 4ULL * 24ULL * 60ULL * 60ULL * 1000ULL; // 4 days
// Upper bound on pooled orphan blocks. Orphans are accepted before the chain-derived difficulty
// check (that lives in Chain_AddBlock, which orphans only reach on attach), so without a cap a
// peer can push blocks at an arbitrary height until the node runs out of memory.
static const size_t MAX_ORPHAN_BLOCKS = 512U;
// A node whose chain tip is older than this many target block times is catching up rather than
// following the tip, and is exempt from the reorg penalty (Horizen does the same via
// IsInitialBlockDownload). Determined purely from local state, so an unverified peer cannot
// trigger the exemption by claiming a large height.
//
// This is also the ONLY way a non-mining node rejoins the network after ending up on a minority
// fork: the penalty is served by local chain growth, and a node that does not mine has no way to
// grow except by adopting the very branch the penalty is gating. It therefore has to be short
// enough that such a node recovers in minutes rather than half a day.
//
// 20 block times is ~30 minutes at a 90s target, far beyond normal Poisson block spacing (a gap
// that long has probability ~e^-20), so a node that is genuinely following the tip will not trip
// it. Note the exemption is all-or-nothing -- once in IBD a node accepts a reorg of any depth --
// so lowering this further widens that hole; it is the number to revisit if deep reorgs ever get
// used against an idle node.
static const uint64_t IBD_TIP_AGE_BLOCKS = 20ULL;
// Number of trailing blocks whose median timestamp is used for the age test above. Using a median
// rather than the tip alone means a single miner cannot backdate one block to fake being in IBD.
static const size_t MEDIAN_TIME_SPAN = 11U;
// Reward schedule acceleration: 1 means normal-speed progression.
#define EMISSION_ACCELERATION_FACTOR 1ULL
@@ -65,32 +148,70 @@ static const double REORG_PENALTY_REF_BLOCK_TIME = 150.0; // reference block tim
// Keep this at 20 to match the canonical curve shape against a 2^64 atomic supply cap.
#define MONERO_EMISSION_SPEED_FACTOR 20U
// Future Autolykos2 constants:
// Autolykos2 epoch / DAG constants.
#define EPOCH_LENGTH 350000 // ~1 year at 90s
#define DAG_BASE_GROWTH (1ULL << 30) // 1 GB per epoch, adjusted by acceleration
//#define DAG_BASE_SIZE (6ULL << 30) // 6 GB, adjusted per cycle based off DAG_BASE_GROWTH
#define DAG_BASE_SIZE (1ULL << 30) // TEMPORARY FOR TESTING
// Swings - calculated as MIN(percentage, absolute GB) to prevent absurd swings from low hashrate or very large DAG growth
#define DAG_MAX_UP_SWING_PERCENTAGE 1.15 // 15%
#define DAG_MAX_DOWN_SWING_PERCENTAGE 0.90 // 10%
#define DAG_MAX_UP_SWING_GB (2ULL << 30) // 2 GB
#define DAG_MAX_DOWN_SWING_GB (1ULL << 30) // 1 GB
#define DAG_GENESIS_SEED 0x00 // Genesis seed is zeroes, every epoch's seed is the hash of the previous block, therefore unpredictable until the block is mined
#define DAG_GENESIS_SEED 0x00 // Epoch 0's seed is all zeroes; epoch k's seed is the hash of the last
// block of epoch k-1, so it is unpredictable until that block is mined.
/**
* Each epoch has 2 phases, connected logarithmically:
* - Phase 1: Aggressive DAG growth (target is ~75% of the max cap) to kick out any ASICs, 30k blocks (roughly 1 month)
* - Phase 2: Stable DAG growth (target is the max cap) to provide a stable environment for GPU miners, 320k blocks (roughly 11 months)
* DAG size band and the miner signal that moves within it.
*
* Growth is the DEFAULT: the size walks up by DAG_EPOCH_STEP every epoch unless miners actively
* brake it. There is deliberately no "grow faster" vote -- every signal a miner can express only
* slows the walk or reverses it. That is what makes the scheme safe against pool capture: under
* stratum-style pooled mining the pool builds the header, so it controls its share of the vote, and
* a pool that wanted a larger DAG to price smaller miners out simply has no lever to pull. The
* entire upward trajectory is set by DAG_EPOCH_STEP and DAG_MAX_SIZE, i.e. by release, not by vote.
*
* DAG_MIN_SIZE is the ASIC-resistance floor: it must stay above the on-die SRAM an ASIC could
* economically carry, because *this constant*, not the vote, is what secures the property. No vote
* outcome can go below it. DAG_MAX_SIZE is the intended destination rather than an emergency bound,
* since the DAG reaches it on its own -- pick it as the largest DAG miners should ever hold.
*
* NOTE: these three sizes are economic judgements, not derivations. Sanity-check them before
* launch. DAG_BASE_SIZE was previously commented as an intended 6 GiB; it now has to sit inside
* the band (see the static_assert below). Lowering the DAG for a test run means lowering
* DAG_MIN_SIZE too, not just DAG_BASE_SIZE.
**/
#define DAG_MIN_SIZE (2ULL << 30) // 2 GiB -- ASIC-resistance floor
#define DAG_BASE_SIZE (2ULL << 30) // epoch 0 size
#define DAG_MAX_SIZE (8ULL << 30) // 8 GiB -- intended destination, ~6 unbraked years from base
#define DAG_EPOCH_STEP (1ULL << 30) // 1 GiB drift per epoch, in either direction
// Vote thresholds as integer numerator/denominator pairs, never float literals: this feeds PoW
// verification, so every node must reach the same verdict. The tests cross-multiply rather than
// divide, so there is no rounding to disagree on.
#define DAG_BRAKE_NUM 1ULL
#define DAG_BRAKE_DEN 2ULL // brake growth when hold+down votes exceed 1/2 of the epoch
#define DAG_DOWN_NUM 7ULL
#define DAG_DOWN_DEN 8ULL // shrink when down votes exceed 7/8 of the epoch, two epochs running
// reserved[0] of the block header carries the vote. 0 must mean GROW: the point of this shape is
// that inaction produces growth, so a miner that knows nothing about the vote contributes to the
// intended default instead of silently freezing the schedule.
#define DAG_VOTE_GROW 0u // default -- let the schedule run
#define DAG_VOTE_HOLD 1u // brake: stop growing
#define DAG_VOTE_DOWN 2u // reverse: shrink (needs a sustained supermajority to take effect)
#define DAG_VOTE_MAX DAG_VOTE_DOWN
static_assert(DAG_MIN_SIZE <= DAG_BASE_SIZE && DAG_BASE_SIZE <= DAG_MAX_SIZE,
"DAG_BASE_SIZE must start inside [DAG_MIN_SIZE, DAG_MAX_SIZE]");
static_assert(DAG_MIN_SIZE % 32ULL == 0ULL && DAG_MAX_SIZE % 32ULL == 0ULL &&
DAG_BASE_SIZE % 32ULL == 0ULL && DAG_EPOCH_STEP % 32ULL == 0ULL,
"Autolykos2 lane addressing requires every DAG size to be a multiple of 32");
static_assert(DAG_EPOCH_STEP > 0ULL, "DAG_EPOCH_STEP must be positive or the DAG can never move");
static const uint64_t M_CAP = 18446744073709551615ULL; // Max uint64
static const uint64_t TAIL_EMISSION = 750000000000ULL; // 0.75 coins per block floor
// No max supply. Instead of halving, it'll follow a more gradual, Monero-like emission curve.
// Phase 3: update once per effective epoch and keep a fixed per-block reward for that epoch.
static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchain_t* chain) {
if (!chain || !chain->blocks) { return 0x00; } // Invalid
size_t height = Chain_Size(chain);
//
// The *AtHeight variants take the height directly and never call Chain_Size/Chain_GetBlockCopy, so
// they are safe to call from inside a chainLock critical section. chainLock is a non-recursive
// pthread_rwlock_t: taking it for reading while this thread already holds it for writing deadlocks
// as soon as another thread is queued for the write lock.
static inline uint64_t GetInflationRateRewardAtHeight(uint256_t currentSupply, uint64_t height) {
const uint64_t effectiveEpochLength =
(EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR) > 0
? (EPOCH_LENGTH / EMISSION_ACCELERATION_FACTOR)
@@ -128,18 +249,20 @@ static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchai
return (currentReward > TAIL_EMISSION) ? currentReward : TAIL_EMISSION;
}
static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_t* chain) {
static inline uint64_t GetInflationRateReward(uint256_t currentSupply, blockchain_t* chain) {
if (!chain || !chain->blocks) { return 0x00; } // Invalid
return GetInflationRateRewardAtHeight(currentSupply, (uint64_t)Chain_Size(chain));
}
static inline uint64_t CalculateBlockRewardAtHeight(uint256_t currentSupply, uint64_t height) {
const uint64_t effectivePhase1Blocks =
(PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR) > 0
? (PHASE1_TARGET_BLOCKS / EMISSION_ACCELERATION_FACTOR)
: 1;
const uint64_t height = (uint64_t)Chain_Size(chain);
// After the phase-one target horizon, only floor/inflation schedule applies.
if (height >= effectivePhase1Blocks) {
return GetInflationRateReward(currentSupply, chain);
return GetInflationRateRewardAtHeight(currentSupply, height);
}
if (currentSupply.limbs[1] > 0 ||
@@ -148,7 +271,7 @@ static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_
currentSupply.limbs[0] >= M_CAP)
{
// Post-Monero phase with unlimited supply: floor/inflation schedule only.
return GetInflationRateReward(currentSupply, chain);
return GetInflationRateRewardAtHeight(currentSupply, height);
}
const uint64_t generated = currentSupply.limbs[0];
@@ -180,80 +303,17 @@ static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_
}
// Phase 2 + 3: floor and epoch inflation updates.
return GetInflationRateReward(currentSupply, chain);
return GetInflationRateRewardAtHeight(currentSupply, height);
}
// Hashing DAG
#include <math.h>
static inline size_t CalculateTargetDAGSize(blockchain_t* chain) {
// Base size plus (base growth * difficulty factor), adjusted by acceleration
if (!chain || !chain->blocks) { return 0; } // Invalid
uint64_t height = (uint64_t)Chain_Size(chain);
if (height < EPOCH_LENGTH) {
return DAG_BASE_SIZE;
static inline uint64_t CalculateBlockReward(uint256_t currentSupply, blockchain_t* chain) {
if (!chain || !chain->blocks) { return 0x00; } // Invalid
return CalculateBlockRewardAtHeight(currentSupply, (uint64_t)Chain_Size(chain));
}
// Get the height - EPOCH_LENGTH block and the last block;
block_t* lastBlock = NULL;
block_t* epochStartBlock = NULL;
if (!Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &lastBlock) || !lastBlock) {
if (lastBlock) Block_Destroy(lastBlock);
return 0;
}
if (!Chain_GetBlockCopy(chain, (size_t)(Chain_Size(chain) - 1 - EPOCH_LENGTH), &epochStartBlock) || !epochStartBlock) {
Block_Destroy(lastBlock);
if (epochStartBlock) Block_Destroy(epochStartBlock);
return 0;
}
int64_t difficultyDelta = (int64_t)epochStartBlock->header.difficultyTarget - (int64_t)lastBlock->header.difficultyTarget;
int64_t growth = (DAG_BASE_GROWTH * difficultyDelta); // Can be negative if difficulty has decreased, which is why we use int64_t
// Clamp
if (growth > 0) {
// Difficulty increased -> Clamp the UPWARD swing
int64_t maxUp = (int64_t)((DAG_BASE_SIZE * 15) / 100); // 15%
if (growth > maxUp) growth = maxUp;
if (growth > (int64_t)DAG_MAX_UP_SWING_GB) growth = DAG_MAX_UP_SWING_GB;
} else {
// Difficulty decreased -> Clamp the DOWNWARD swing
int64_t maxDown = (int64_t)((DAG_BASE_SIZE * 10) / 100); // 10%
if (-growth > maxDown) growth = -maxDown;
if (-growth > (int64_t)DAG_MAX_DOWN_SWING_GB) growth = -(int64_t)DAG_MAX_DOWN_SWING_GB;
}
int64_t targetSize = (int64_t)DAG_BASE_SIZE + growth;
if (targetSize <= 0) {
Block_Destroy(lastBlock);
Block_Destroy(epochStartBlock);
return 0;
}
size_t out = (size_t)targetSize;
Block_Destroy(lastBlock);
Block_Destroy(epochStartBlock);
return out;
}
static inline void GetNextDAGSeed(blockchain_t* chain, uint8_t outSeed[32]) {
if (!chain || !chain->blocks || !outSeed) { return; } // Invalid
uint64_t height = (uint64_t)Chain_Size(chain);
if (height < EPOCH_LENGTH) {
memset(outSeed, DAG_GENESIS_SEED, 32);
return;
}
block_t* prevBlock = NULL;
if (!Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &prevBlock) || !prevBlock) {
memset(outSeed, 0x00, 32); // Fallback to zeroes if we can't get the previous block for some reason; The caller should treat this as an error if height >= EPOCH_LENGTH
if (prevBlock) Block_Destroy(prevBlock);
return;
}
Block_CalculateHash(prevBlock, outSeed);
Block_Destroy(prevBlock);
}
// Hashing DAG: see Chain_DagParamsForHeight in block/chain.h. Both the size and the epoch seed are
// derived from the chain by that one function, so the mining and verification paths cannot drift
// apart. The previous CalculateTargetDAGSize/GetNextDAGSeed pair lived here, took chainLock
// internally, was not epoch-aligned, and disagreed with the verifier's own copy in main.c.
#endif
include/khash/khash.h
-1
View File
@@ -113,7 +113,6 @@ int main() {
* Added destructor
*/
#ifndef __AC_KHASH_H
#define __AC_KHASH_H
include/nets/net_node.h
+34 -3
View File
@@ -25,6 +25,7 @@ typedef struct node_discovery node_discovery_t;
#include <block/block.h>
#include <block/chain.h>
#include <block/transaction.h>
#include <stdatomic.h>
typedef struct {
tcp_server_t* server;
@@ -42,7 +43,10 @@ typedef struct {
void* callbackUser;
// Maintenance thread for periodic tasks (orphan attach, pruning, metrics)
pthread_t maintenanceThread;
volatile int maintenanceRunning;
// Cross-thread stop flag: written by Node_Destroy on the main thread, read by the maintenance
// thread's loop condition. `volatile` stops the compiler hoisting the load but provides neither
// atomicity nor ordering, so this has to be a real atomic (and TSan rightly flagged it).
_Atomic int maintenanceRunning;
int maintenanceIntervalMs;
// UDP ping/pong daemon (latency oracle) and peer discovery state
udp_node_t* udpNode;
@@ -68,6 +72,29 @@ int Node_BroadcastTransaction(net_node_t* node, signed_transaction_t* tx, tcp_co
// Helpers for outbound peer selection and block broadcast
int Node_GetBestOutboundPeer(net_node_t* node, tcp_connection_t** outConn, uint64_t* outHeight);
/**
* Delivery receipts for windowed sync.
*
* A FETCH_BLOCK reply is handled on the peer's io thread and may legitimately never reach the
* chain: a block belonging to a competing branch is filed in the orphan pool instead. A sync loop
* that infers arrival from the chain growing therefore cannot tell "arrived but forked" from "lost
* in transit", so it re-requests until it times out. Against a peer on a fork that costs one full
* retry-and-timeout cycle for EVERY block, which is why syncing to a forked peer used to crawl.
*
* DUPLICATE is what makes a backwards fork walk terminate: it means we already hold exactly that
* block, so the two chains agree at that height and there is no reason to keep descending.
**/
typedef enum {
NODE_DELIVERY_APPENDED = 0, // joined our chain
NODE_DELIVERY_DUPLICATE = 1, // we already held this exact block -- common ground
NODE_DELIVERY_ORPHANED = 2, // belongs to a competing branch; now in the orphan pool
NODE_DELIVERY_REJECTED = 3 // failed validation
} node_delivery_status_t;
void Node_NoteBlockDelivered(uint64_t height, node_delivery_status_t status);
bool Node_TakeBlockDelivery(uint64_t height, node_delivery_status_t* outStatus);
void Node_ResetBlockDeliveries(void);
void Node_BroadcastChainRange(net_node_t* node, size_t startHeightInclusive, tcp_connection_t* sourceConn);
// Callback logic
@@ -85,8 +112,12 @@ void Node_GetClientList(net_node_t* node, tcp_connection_t** outClients, size_t*
// Returns non-zero on success (usable endpoint with a known, non-zero port), zero otherwise.
int Node_ConnListenEndpoint(const tcp_connection_t* conn, struct sockaddr_storage* out);
// Returns the node identity advertised by a connection's peer, or 0 if it is not known yet.
uint64_t Node_ConnPeerNodeId(const tcp_connection_t* conn);
// Fills outEndpoints with the listen endpoints of all current connections (inbound + outbound),
// deduped by IP+port. Returns the number of endpoints written (<= maxOut).
size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpoints, size_t maxOut);
// deduped by IP+port, and outNodeIds (optional, may be NULL) with the matching peer identities.
// Returns the number of endpoints written (<= maxOut).
size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpoints, uint64_t* outNodeIds, size_t maxOut);
#endif
include/nets/nodediscovery.h
+16
View File
@@ -23,6 +23,22 @@ void NodeDiscovery_OnGetPeers(node_discovery_t* disc, tcp_connection_t* fromConn
// Decode a received PEERS payload and fold a couple of its endpoints into the known-peer table.
void NodeDiscovery_OnPeersReceived(node_discovery_t* disc, tcp_connection_t* fromConn, const unsigned char* payload, size_t payloadLen);
// Strike a peer (by its listen endpoint) from the known-peer table. Called when a peer becomes
// logically disconnected (no remaining connection to it).
void NodeDiscovery_RemovePeer(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Record the node identity behind an endpoint (learned from a completed HELLO/ACK_HELLO). Entries
// carrying an identity we are already connected to are skipped by the connect picker, which is what
// stops a multi-homed peer from being dialed once per address it is reachable on.
void NodeDiscovery_NoteIdentity(node_discovery_t* disc, const struct sockaddr_storage* endpoint, uint64_t nodeId);
// Mark an endpoint as one of our own, permanently. Self endpoints are never added to the known-peer
// table, never pinged and never dialed. Seeded from the local interface addresses at creation and
// extended whenever a handshake turns out to come from ourselves.
void NodeDiscovery_MarkSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Returns non-zero if the endpoint is known to be one of our own.
int NodeDiscovery_IsSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint);
// Dump the known-peer table to stdout (for the CLI `peers` command).
void NodeDiscovery_PrintPeers(node_discovery_t* disc);
include/nets/orphan_pool.h
+25 -3
View File
@@ -2,6 +2,7 @@
#define ORPHAN_POOL_H
#include <stdint.h>
#include <stdbool.h>
#include <block/block.h>
#include <block/chain.h>
@@ -10,11 +11,32 @@ void OrphanPool_Init(void);
void OrphanPool_Destroy(void);
// Insert an orphan block into the pool. Ownership of `block` is transferred to the pool.
// `height` is the block number from the header.
void OrphanPool_Insert(block_t* block, uint64_t height);
// `height` is the block number from the header. `observedAtTipHeight` is the local chain tip
// height at the moment the block arrived; it is stamped once and drives the Horizen reorg
// penalty, so it must never be re-derived from a later tip.
// Duplicates (same block hash) are rejected and the block is destroyed.
void OrphanPool_Insert(block_t* block, uint64_t height, uint64_t observedAtTipHeight);
// Attempt to attach any orphans whose parents now exist in `chain`.
// Attempt to attach any orphans whose parents now exist in `chain`, and to adopt a competing
// branch when one is heavier and has served its reorg penalty.
// Returns the number of blocks successfully attached.
size_t OrphanPool_AttemptAttach(blockchain_t* chain);
/**
* As OrphanPool_AttemptAttach, but skips the reorg delay penalty when `bypassPenalty` is set.
*
* Reserved for an explicit operator action (`sync force`). The penalty is served by local chain
* growth, so a node that is neither mining nor stale enough to count as catching up can never
* clear it by itself; this is the manual way out for an operator who knows their branch is the
* wrong one. Work comparison and linkage still apply, so it cannot adopt a lighter branch, and
* nothing a peer sends can reach it.
**/
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty);
// True if a block with this hash is already pooled.
bool OrphanPool_Contains(const uint8_t blockHash[32]);
// Number of pooled orphans (diagnostics).
size_t OrphanPool_Size(void);
#endif
include/numgen.h
+5
View File
@@ -11,4 +11,9 @@ uint16_t random_two_byte(void);
uint32_t random_four_byte(void);
uint64_t random_eight_byte(void);
// Draws from the OS entropy pool instead of the srand()-seeded generator, which repeats across
// processes started within the same second. Use this wherever a value must be unique between nodes
// (e.g. the node identity). Never returns 0.
uint64_t random_secure_eight_byte(void);
#endif
include/runtime_state.h
+4
View File
@@ -17,6 +17,10 @@ extern const char* chainDataDir;
extern unsigned short listenPort;
extern bool echoPeersEnabled;
extern bool forceOrphanReorgEnabled;
// Random per-run identity of this node, advertised in HELLO/ACK_HELLO. A host can be reachable
// under many addresses (especially over IPv6), so an (ip, port) endpoint is not a peer identity:
// this nonce is what lets us recognise our own connections and a peer we already talk to.
extern uint64_t localNodeId;
// Global synchronization primitives for runtime state
extern pthread_rwlock_t chainLock; // protects chain structure and related mutations
include/tcpd/tcpconnection.h
+16
View File
@@ -3,6 +3,7 @@
#include <arpa/inet.h>
#include <pthread.h>
#include <stdatomic.h>
#include <stdbool.h>
#include <stddef.h>
#include <stdint.h>
@@ -30,6 +31,11 @@ struct tcp_connection_t {
// For OUTBOUND connections the peerAddr port already is the listen port; this matters for INBOUND peers.
uint16_t peerListenPort;
// Peer's advertised node identity (learned from HELLO/ACK_HELLO). 0 until known / peer too old
// to advertise one. Unlike the peer address, this is stable across all of a multi-homed peer's
// endpoints, so it is what identifies the node behind this connection.
uint64_t peerNodeId;
pthread_t ioThread;
pthread_mutex_t sendLock;
pthread_mutex_t stateLock;
@@ -37,6 +43,11 @@ struct tcp_connection_t {
bool closing;
bool disconnectedNotified;
// Non-zero while another thread holds a raw pointer to this connection taken from a
// lock-protected snapshot and used after releasing the lock. The reaper must not free a
// pinned connection. See TcpConnection_Pin/Unpin.
atomic_int pinCount;
unsigned char* dataBuf;
size_t dataBufLen;
size_t dataBufCap;
@@ -75,4 +86,9 @@ void TcpConnection_RequestClose(tcp_connection_t* conn);
void TcpConnection_MarkDisconnectNotified(tcp_connection_t* conn);
bool TcpConnection_IsDisconnectNotified(tcp_connection_t* conn);
// Pin/unpin a connection so a background reaper won't free it while a caller still holds a raw
// pointer to it (e.g. across a blocking operation after releasing the collection lock).
void TcpConnection_Pin(tcp_connection_t* conn);
void TcpConnection_Unpin(tcp_connection_t* conn);
#endif
include/tcpd/tcpserver.h
+4 -1
View File
@@ -7,12 +7,15 @@
#include <constants.h>
#include <tcpd/tcpconnection.h>
#include <stdatomic.h>
typedef struct {
int sockFd; // IPv6 listening socket (-1 if IPv6 unavailable)
int sockFdV4; // IPv4 listening socket (-1 on bind failure)
int opt;
int isRunning;
// Cross-thread stop flag: cleared by TcpServer_Stop, read by both accept threads and by
// exiting client threads. Must be atomic, not a plain int.
_Atomic int isRunning;
void* owner;
// Called before the client thread runs
include/txmempool.h
+29 -1
View File
@@ -2,11 +2,20 @@
#define TXMEMPOOL_H
#include <block/transaction.h>
#include <khash/khash.h>
#include <khash.h>
#include <utils.h>
#include <uint256.h>
// See balance_sheet.h: khash's macro expansion is not -Wconversion clean, and
// -isystem does not cover code expanded from a system header's macros.
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wconversion"
#endif
KHASH_INIT(tx_mempool_map_m, key32_t, signed_transaction_t, 1, hash_key32, eq_key32)
#if defined(__GNUC__) || defined(__clang__)
#pragma GCC diagnostic pop
#endif
extern khash_t(tx_mempool_map_m)* txMempool;
void TxMempool_Init();
@@ -17,6 +26,25 @@ bool TxMempool_Snapshot(signed_transaction_t** outTxs, size_t* outCount);
void TxMempool_Print();
// Remove a transaction from the mempool by its hash. Returns true if removed.
bool TxMempool_Remove(const uint8_t* txHash);
/**
* Admission policy: should this transaction be held and relayed?
*
* LOCAL POLICY, NOT CONSENSUS. A block containing a transaction this rejects is still accepted --
* see TX_MAX_FUTURE_DRIFT_MS / TX_EXPIRY_MS in constants.h for why the two are kept apart.
*
* Both bounds are measured against the node's own clock, NOT against the chain tip's timestamp.
* Measuring "future" against the last block assumes blocks keep arriving: on a quiet chain the tip
* can be hours old, and an honest transaction created right now would look hours ahead of it and be
* refused. Sending would become impossible exactly when the chain is idle.
*
* Deliberately NOT applied when a rollback returns transactions to the pool: those were already in
* the chain, so they are legitimate by definition and must not be dropped for looking old.
**/
bool TxMempool_PolicyAccepts(const signed_transaction_t* tx, uint64_t nowMs);
// Drop transactions older than TX_EXPIRY_MS. Returns how many were removed.
size_t TxMempool_PruneExpired(uint64_t nowMs);
void TxMempool_Destroy();
#endif
include/udpd/udpnode.h
+4 -1
View File
@@ -7,6 +7,7 @@
#include <netinet/in.h>
#include <udpd/udppackettype.h>
#include <stdatomic.h>
#define UDP_LISTEN_PORT 9393
#define UDP_PING_RETRY_INTERVAL_MS 1000
@@ -25,7 +26,9 @@ typedef struct udp_node {
int sockFd; // AF_INET6, IPV6_V6ONLY=1
int sockFdV4; // AF_INET
volatile int isRunning;
// Cross-thread stop flag: cleared by UdpNode_Stop, read by the recv and retry thread loops.
// See the note on net_node_t.maintenanceRunning -- volatile is not a substitute for atomic.
_Atomic int isRunning;
pthread_t recvThreadV6;
pthread_t recvThreadV4;
include/uint256.h
+66
View File
@@ -114,6 +114,72 @@ static inline int uint256_cmp(const uint256_t* a, const uint256_t* b) {
return 0;
}
static inline bool uint256_is_zero(const uint256_t* a) {
return a && a->limbs[0] == 0 && a->limbs[1] == 0 && a->limbs[2] == 0 && a->limbs[3] == 0;
}
/**
* Builds a uint256 from 32 big-endian bytes, the layout used by hashes and by decoded
* difficulty targets (see DecodeCompactTarget).
**/
static inline uint256_t uint256_from_be_bytes(const uint8_t bytes[32]) {
uint256_t res = {{0, 0, 0, 0}};
if (!bytes) {
return res;
}
for (int limb = 0; limb < 4; ++limb) {
// limbs[0] is the least significant, so it holds the LAST eight bytes.
const uint8_t* src = bytes + (3 - limb) * 8;
uint64_t value = 0;
for (int b = 0; b < 8; ++b) {
value = (value << 8) | (uint64_t)src[b];
}
res.limbs[limb] = value;
}
return res;
}
static inline void uint256_bitwise_not(uint256_t* a) {
if (!a) {
return;
}
for (int i = 0; i < 4; ++i) {
a->limbs[i] = ~a->limbs[i];
}
}
/**
* Unsigned 256-bit division by restoring binary long division.
* Returns false (leaving *outQuotient untouched) when dividing by zero.
**/
static inline bool uint256_divide(const uint256_t* numerator, const uint256_t* denominator, uint256_t* outQuotient) {
if (!numerator || !denominator || !outQuotient || uint256_is_zero(denominator)) {
return false;
}
uint256_t quotient = uint256_from_u64(0);
uint256_t remainder = uint256_from_u64(0);
for (int bit = 255; bit >= 0; --bit) {
// remainder = (remainder << 1) | bit_of_numerator
for (int i = 3; i > 0; --i) {
remainder.limbs[i] = (remainder.limbs[i] << 1) | (remainder.limbs[i - 1] >> 63);
}
remainder.limbs[0] <<= 1;
remainder.limbs[0] |= (numerator->limbs[bit / 64] >> (bit % 64)) & 1ULL;
if (uint256_cmp(&remainder, denominator) >= 0) {
(void)uint256_subtract(&remainder, denominator);
quotient.limbs[bit / 64] |= (1ULL << (bit % 64));
}
}
*outQuotient = quotient;
return true;
}
static inline void uint256_serialize(const uint256_t* value, char* out) {
if (!value || !out) {
return;
include/utils.h
+29 -34
View File
@@ -9,6 +9,9 @@
#include <crypto/crypto.h>
#include <uint256.h>
#include <time.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
typedef struct {
uint8_t bytes[32];
@@ -252,55 +255,47 @@ static inline bool ParseHexAddress32(const char* in, uint8_t outAddress[32]) {
}
static inline bool IsValidIPv4(const char* ip) {
struct addrinfo hints, *res;
int status;
if (!ip || *ip == '\0') {
return false;
}
int octetCount = 0;
const char* p = ip;
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_INET; // Only IPv4
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_NUMERICHOST; // Only numeric addresses, no DNS lookups
while (*p != '\0') {
if (octetCount >= 4) {
status = getaddrinfo(ip, NULL, &hints, &res);
if (status == 0) {
freeaddrinfo(res);
return true;
}
return false;
}
if (*p < '0' || *p > '9') {
static inline bool IsValidIPv6(const char* ip) {
struct addrinfo hints, *res;
int status;
if (!ip || *ip == '\0') {
return false;
}
unsigned int value = 0;
int digits = 0;
while (*p >= '0' && *p <= '9') {
value = (value * 10u) + (unsigned int)(*p - '0');
if (value > 255u) {
return false;
}
++digits;
if (digits > 3) {
return false;
}
++p;
}
memset(&hints, 0, sizeof hints);
hints.ai_family = AF_INET6; // Only IPv6
hints.ai_socktype = SOCK_STREAM;
hints.ai_flags = AI_NUMERICHOST; // Only numeric addresses, no DNS lookups
if (digits == 0) {
status = getaddrinfo(ip, NULL, &hints, &res);
if (status == 0) {
freeaddrinfo(res);
return true;
}
return false;
}
++octetCount;
if (octetCount < 4) {
if (*p != '.') {
return false;
}
++p;
if (*p == '\0') {
return false;
}
}
}
return octetCount == 4;
}
static inline void Uint256ToDecimal(const uint256_t* value, char* out, size_t outSize) {
if (!value || !out || outSize == 0) {
return;
src/autolykos2/autolykos2.c
-31
View File
@@ -409,37 +409,6 @@ bool Autolykos2_Hash(
);
}
bool Autolykos2_LightHash(const uint8_t* seed, blockchain_t* chain, uint64_t nonce, uint8_t* out) {
if (!seed || !chain || !out) {
return false;
}
const uint64_t height = (uint64_t)Chain_Size(chain);
const size_t dagBytes = CalculateTargetDAGSize(chain);
if (dagBytes < 32 || (dagBytes % 32) != 0) {
return false;
}
const size_t laneCount64 = dagBytes / 32u;
if (laneCount64 == 0 || laneCount64 > UINT32_MAX) {
return false;
}
// Light path derives the needed DAG lanes from seed on-demand, no large DAG allocation required.
return Autolykos2_HashCore(
seed,
seed,
seed,
32,
nonce,
height,
(uint32_t)laneCount64,
NULL,
false,
out
);
}
bool Autolykos2_LightHashAtHeight(
const uint8_t seed32[32],
const uint8_t* message,
src/block/block.c
+150 -42
View File
@@ -1,45 +1,130 @@
#include <block/block.h>
#include <block/chain.h>
#include <autolykos2/autolykos2.h>
#include <utils.h>
#include <stdlib.h>
#include <pthread.h>
/**
* The process-global mining DAG.
*
* Guarded by `g_powCtxLock` because generation frees and reallocates the buffer that hashing reads
* from: without the lock, an epoch rollover would pull the DAG out from under a miner mid-hash.
* Only the miner ever builds or reads this -- validation goes through the light path -- so the lock
* is essentially uncontended, and a node that does not mine never allocates a DAG at all.
**/
static Autolykos2Context* g_autolykos2Ctx = NULL;
static pthread_mutex_t g_powCtxLock = PTHREAD_MUTEX_INITIALIZER;
static uint64_t g_dagEpoch = 0;
// The seed the current DAG was generated from. Matching on epoch index and size is NOT enough: a
// reorg replaces the block an epoch's seed is derived from while leaving the epoch index and size
// unchanged, so a stale DAG would still look current and silently hash against the wrong lanes.
static uint8_t g_dagSeed[32];
static bool g_dagReady = false;
static Autolykos2Context* GetAutolykos2Ctx(void) {
// Caller must hold `g_powCtxLock`.
static Autolykos2Context* GetAutolykos2CtxLocked(void) {
if (!g_autolykos2Ctx) {
g_autolykos2Ctx = Autolykos2_Create();
if (!g_autolykos2Ctx) {
fprintf(stderr, "Failed to create Autolykos2 context\n");
exit(1);
}
Autolykos2_DagAllocate(g_autolykos2Ctx, DAG_BASE_SIZE);
// Deliberately no DagAllocate here. Allocating without generating leaves dag.len == 0, so
// every heavy hash fails -- which used to be indistinguishable from a valid proof, because
// the failure path handed back a zeroed hash that compares below every target.
}
return g_autolykos2Ctx;
}
void Block_ShutdownPowContext(void) {
pthread_mutex_lock(&g_powCtxLock);
if (g_autolykos2Ctx) {
Autolykos2_Destroy(g_autolykos2Ctx);
g_autolykos2Ctx = NULL;
}
g_dagReady = false;
pthread_mutex_unlock(&g_powCtxLock);
}
bool Block_RebuildAutolykos2Dag(size_t dagBytes, const uint8_t seed32[32]) {
if (!seed32 || dagBytes == 0) {
bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8_t seed32[32]) {
if (!seed32 || dagBytes < 32u || (dagBytes % 32u) != 0u) {
return false;
}
Autolykos2Context* ctx = GetAutolykos2Ctx();
if (!ctx) {
return false;
pthread_mutex_lock(&g_powCtxLock);
// Already built from exactly this seed at this size: generation is seconds of work, never redo
// it. The seed has to be part of the test -- see g_dagSeed.
if (g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0) {
pthread_mutex_unlock(&g_powCtxLock);
return true;
}
Autolykos2Context* ctx = GetAutolykos2CtxLocked();
g_dagReady = false; // the buffer is about to be invalid; no heavy hash may run against it
// Generation is one Blake2b per 64 bytes, single-threaded, so a multi-GiB DAG is tens of
// seconds. Say so rather than leaving the miner looking hung.
printf("Generating the epoch %llu mining DAG (%zu MiB), this takes a moment...\n",
(unsigned long long)epochIndex, dagBytes >> 20);
fflush(stdout);
Autolykos2_DagClear(ctx);
if (!Autolykos2_DagAllocate(ctx, dagBytes)) {
const bool ok = Autolykos2_DagAllocate(ctx, dagBytes) && Autolykos2_DagGenerate(ctx, seed32);
if (ok) {
g_dagEpoch = epochIndex;
memcpy(g_dagSeed, seed32, 32);
g_dagReady = true;
}
pthread_mutex_unlock(&g_powCtxLock);
return ok;
}
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]) {
if (!block || !seed32 || !outHash) {
return false;
}
return Autolykos2_DagGenerate(ctx, seed32);
pthread_mutex_lock(&g_powCtxLock);
// Verifying the SEED here, not just the epoch and size, is what makes this impossible to
// misuse. A reorg changes the block an epoch's seed is derived from while the epoch index and
// size stay put, so an epoch+size check alone happily accepts a DAG built from the pre-reorg
// seed and returns a hash for the wrong lanes -- which shows up as a valid block failing PoW
// while a branch is being applied. A mismatch yields false and the caller derives the lanes
// from the seed instead.
const bool usable = g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0;
const bool ok = usable &&
Autolykos2_Hash(
g_autolykos2Ctx,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
block->header.blockNumber, // full 64-bit width; the light path takes uint64
outHash);
pthread_mutex_unlock(&g_powCtxLock);
return ok;
}
bool Block_PowHashLight(const block_t* block, size_t dagBytes, const uint8_t seed32[32], uint8_t outHash[32]) {
if (!block || !seed32 || !outHash) {
return false;
}
return Autolykos2_LightHashAtHeight(
seed32,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
block->header.blockNumber,
dagBytes,
outHash);
}
block_t* Block_Create() {
@@ -133,30 +218,6 @@ void Block_CalculateMerkleRoot(const block_t* block, uint8_t* outHash) {
free(next);
}
void Block_CalculateAutolykos2Hash(const block_t* block, uint8_t* outHash) {
if (!block || !outHash) {
return;
}
// PoW hash is computed from the block header, while canonical block hash remains SHA256.
Autolykos2Context* ctx = GetAutolykos2Ctx();
if (!ctx) {
memset(outHash, 0, 32);
return;
}
if (!Autolykos2_Hash(
ctx,
(const uint8_t*)&block->header,
sizeof(block_header_t),
block->header.nonce,
(uint32_t)block->header.blockNumber,
outHash
)) {
memset(outHash, 0, 32);
}
}
void Block_AddTransaction(block_t* block, signed_transaction_t* tx) {
if (!block || !tx || !block->transactions) {
return;
@@ -189,7 +250,8 @@ static int Uint256_CompareBE(const uint8_t a[32], const uint8_t b[32]) {
return 0;
}
bool Block_HasValidProofOfWork(const block_t* block) {
bool Block_HasValidProofOfWorkWithParams(const block_t* block, uint64_t epochIndex,
size_t dagBytes, const uint8_t seed32[32]) {
if (!block) {
return false;
}
@@ -199,12 +261,49 @@ bool Block_HasValidProofOfWork(const block_t* block) {
return false;
}
// Prefer the prebuilt DAG when it is provably the one for this block's epoch and size -- the
// miner keeps it warm, and reading a lane beats recomputing it -- otherwise derive the lanes
// from the epoch seed. The two produce identical hashes, so which one runs is invisible to
// consensus; only speed differs.
uint8_t hash[32];
Block_CalculateAutolykos2Hash(block, hash);
if (!Block_PowHashHeavy(block, epochIndex, dagBytes, seed32, hash) &&
!Block_PowHashLight(block, dagBytes, seed32, hash)) {
// Fail CLOSED. This used to hand back a zeroed hash on any failure and compare that to the
// target -- and zero is below every target, so a DAG that was missing, mis-sized or failed
// to build made the PoW check pass for every block instead of rejecting them.
return false;
}
return Uint256_CompareBE(hash, target) <= 0;
}
bool Block_HasValidProofOfWork(const block_t* block, blockchain_t* chain) {
if (!block || !chain) {
return false;
}
size_t dagBytes = 0;
uint8_t seed[32];
if (!Chain_DagParamsForHeight(chain, block->header.blockNumber, &dagBytes, seed)) {
return false;
}
const uint64_t epochIndex = block->header.blockNumber / (uint64_t)EPOCH_LENGTH;
return Block_HasValidProofOfWorkWithParams(block, epochIndex, dagBytes, seed);
}
bool Block_HasValidVote(const block_t* block) {
if (!block) {
return false;
}
// Unrecognised vote values and non-zero spare bytes are rejected rather than ignored, so the
// header has no bits whose meaning is undefined and nothing to grind for extra nonce space.
return block->header.reserved[0] <= (uint8_t)DAG_VOTE_MAX &&
block->header.reserved[1] == 0u &&
block->header.reserved[2] == 0u;
}
bool Block_AllTransactionsValid(const block_t* block) {
if (!block || !block->transactions) {
return false;
@@ -296,15 +395,24 @@ bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinba
return true;
}
bool Block_IsFullyValid(const block_t* block) {
bool merkleValid = false;
uint8_t calculatedMerkleRoot[32];
if (block && block->transactions) {
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
merkleValid = (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) == 0);
bool Block_HasValidStructure(const block_t* block) {
if (!block || !block->transactions) {
return false;
}
return Block_HasValidProofOfWork(block) && Block_AllTransactionsValid(block) && DynArr_size(block->transactions) > 0 && merkleValid;
uint8_t calculatedMerkleRoot[32];
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
if (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) != 0) {
return false;
}
return Block_HasValidVote(block) &&
Block_AllTransactionsValid(block) &&
DynArr_size(block->transactions) > 0;
}
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain) {
return Block_HasValidStructure(block) && Block_HasValidProofOfWork(block, chain);
}
void Block_Destroy(block_t* block) {
src/block/chain.c
+1097 -63
View File
File diff suppressed because it is too large Load Diff
src/main.c
+592 -215
View File
File diff suppressed because it is too large Load Diff
src/nets/fetch_scheduler.c
+41 -12
View File
@@ -1,24 +1,53 @@
#include <nets/fetch_scheduler.h>
#include <constants.h>
#include <math.h>
// Note: floating point is used intentionally here for readability and
// because the final penalty is rounded to whole blocks. This keeps the
// implementation straightforward while avoiding subtle integer overflow
// for large exponents. If desired, replace with fixed-point arithmetic.
// Integer-only on purpose. This penalty gates fork choice (see Chain_ReplaceBranch), so every node
// must compute the exact same number of blocks from the same reorg depth. The previous
// implementation used double/pow/ceil, which is not reproducible across platforms and compilers.
uint64_t FetchScheduler_ComputeReorgPenaltyBlocks(uint64_t delayBlocks) {
if (delayBlocks <= REORG_PENALTY_GRACE_BLOCKS) {
return 0ULL;
}
double B = (double)delayBlocks;
double factor = REORG_PENALTY_FACTOR;
double exp = REORG_PENALTY_EXPONENT;
double timeScale = ((double)TARGET_BLOCK_TIME) / REORG_PENALTY_REF_BLOCK_TIME;
uint64_t depth = delayBlocks;
if (depth > REORG_PENALTY_MAX_DEPTH) {
depth = REORG_PENALTY_MAX_DEPTH;
}
double raw = factor * pow(B, exp) * timeScale;
if (raw < 0.0) raw = 0.0;
// depth^EXPONENT, saturating rather than wrapping.
uint64_t raised = 1ULL;
for (uint32_t i = 0; i < REORG_PENALTY_EXPONENT; ++i) {
if (depth != 0ULL && raised > UINT64_MAX / depth) {
return UINT64_MAX;
}
raised *= depth;
}
// Scale by theta and by the block-time ratio, as one fraction so there is a single rounding
// step: penalty = ceil(raised * FACTOR_NUM * REF_BLOCK_TIME / (FACTOR_DEN * TARGET_BLOCK_TIME))
//
// REF_BLOCK_TIME is the NUMERATOR and TARGET_BLOCK_TIME the DENOMINATOR, not the other way
// round. The result is a count of BLOCKS, so the wall-clock protection it buys is
// penalty(d) * TARGET_BLOCK_TIME ~= d^p * REF_BLOCK_TIME -- TARGET_BLOCK_TIME cancels, and the
// protection is the same number of seconds whatever the block time is. Inverting these two
// makes wall-clock protection scale as TARGET_BLOCK_TIME^2, so shortening the block time
// silently weakens reorg protection. Do not "simplify" this back.
const uint64_t numeratorScale = REORG_PENALTY_FACTOR_NUM * REORG_PENALTY_REF_BLOCK_TIME;
const uint64_t denominator = REORG_PENALTY_FACTOR_DEN * (uint64_t)TARGET_BLOCK_TIME;
if (denominator == 0ULL) {
return 0ULL;
}
if (numeratorScale != 0ULL && raised > UINT64_MAX / numeratorScale) {
return UINT64_MAX;
}
const uint64_t numerator = raised * numeratorScale;
// Ceiling division without overflowing on the +denominator-1 term.
uint64_t penalty = numerator / denominator;
if (numerator % denominator != 0ULL) {
penalty++;
}
uint64_t penalty = (uint64_t)ceil(raw);
return penalty;
}
src/nets/net_node.c
+544 -88
View File
@@ -102,13 +102,15 @@ static int Node_HasOutboundTo(net_node_t* node, const struct sockaddr_storage* e
// Returns non-zero if some inbound connection OTHER than `self` already has the given listen
// endpoint (used to reject a duplicate inbound once we learn the peer's advertised listen port).
// Connections that are already tearing down do not count - otherwise a peer reconnecting from the
// same endpoint gets its fresh inbound rejected by the corpse of the previous one.
static int Node_HasOtherInboundFrom(net_node_t* node, const tcp_connection_t* self, const struct sockaddr_storage* endpoint) {
if (!node->server) return 0;
int found = 0;
pthread_mutex_lock(&node->server->clientsMutex);
for (size_t i = 0; i < node->server->maxClients; ++i) {
tcp_connection_t* other = node->server->clientsArrPtr ? node->server->clientsArrPtr[i] : NULL;
if (!other || other == self) continue;
if (!other || other == self || TcpConnection_IsDisconnectNotified(other)) continue;
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(other, &ep) && NetNode_EndpointEqual(&ep, endpoint)) {
found = 1;
@@ -119,6 +121,80 @@ static int Node_HasOtherInboundFrom(net_node_t* node, const tcp_connection_t* se
return found;
}
// Returns non-zero if a live connection OTHER than `self` with the same role already belongs to the
// node identified by nodeId. This is the endpoint-independent duplicate check: a multi-homed peer
// reaches us from several addresses, so comparing endpoints alone lets the same node in twice.
static int Node_HasOtherConnectionToNode(net_node_t* node, const tcp_connection_t* self, uint64_t nodeId) {
if (nodeId == 0) return 0;
int found = 0;
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS && !found; ++i) {
tcp_connection_t* c = node->outboundClients[i].connection;
if (!c || c == self || TcpConnection_IsDisconnectNotified(c)) continue;
if (c->role == self->role && c->peerNodeId == nodeId) found = 1;
}
pthread_mutex_unlock(&node->outboundLock);
if (found) return 1;
if (node->server) {
pthread_mutex_lock(&node->server->clientsMutex);
for (size_t i = 0; i < node->server->maxClients && !found; ++i) {
tcp_connection_t* c = node->server->clientsArrPtr ? node->server->clientsArrPtr[i] : NULL;
if (!c || c == self || TcpConnection_IsDisconnectNotified(c)) continue;
if (c->role == self->role && c->peerNodeId == nodeId) found = 1;
}
pthread_mutex_unlock(&node->server->clientsMutex);
}
return found;
}
// Returns non-zero if a connection OTHER than `exclude` to the same peer is still live - matched
// either on the listen endpoint or, when known, on the peer's identity (which also covers its other
// addresses). A connection that is itself mid-disconnect (disconnectedNotified) does not count as
// live - this is what lets us decide a peer is fully gone even when both its inbound and outbound
// drop simultaneously.
static int Node_HasLiveConnectionTo(net_node_t* node, const struct sockaddr_storage* endpoint, uint64_t nodeId, const tcp_connection_t* exclude) {
int found = 0;
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS && !found; ++i) {
tcp_connection_t* c = node->outboundClients[i].connection;
if (!c || c == exclude || TcpConnection_IsDisconnectNotified(c)) continue;
if (nodeId != 0 && c->peerNodeId == nodeId) { found = 1; break; }
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(c, &ep) && NetNode_EndpointEqual(&ep, endpoint)) found = 1;
}
pthread_mutex_unlock(&node->outboundLock);
if (found) return 1;
if (node->server) {
pthread_mutex_lock(&node->server->clientsMutex);
for (size_t i = 0; i < node->server->maxClients && !found; ++i) {
tcp_connection_t* c = node->server->clientsArrPtr ? node->server->clientsArrPtr[i] : NULL;
if (!c || c == exclude || TcpConnection_IsDisconnectNotified(c)) continue;
if (nodeId != 0 && c->peerNodeId == nodeId) { found = 1; break; }
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(c, &ep) && NetNode_EndpointEqual(&ep, endpoint)) found = 1;
}
pthread_mutex_unlock(&node->server->clientsMutex);
}
return found;
}
// Called when a connection to a peer drops. Strikes the peer from the discovery table, but only
// once it is logically disconnected - i.e. no other live connection (inbound or outbound) to the
// same node remains. Must be called from the disconnect callback while `conn` is still valid and
// outside outboundLock/clientsMutex.
static void Node_HandlePeerDisconnect(net_node_t* node, tcp_connection_t* conn) {
if (!node || !node->discovery || !conn) return;
struct sockaddr_storage ep;
if (!Node_ConnListenEndpoint(conn, &ep)) return; // never advertised an endpoint -> not tracked
// Still reachable via another connection (possibly on one of its other addresses).
if (Node_HasLiveConnectionTo(node, &ep, conn->peerNodeId, conn)) return;
NodeDiscovery_RemovePeer(node->discovery, &ep);
}
int Node_ConnListenEndpoint(const tcp_connection_t* conn, struct sockaddr_storage* out) {
if (!conn || !out) return 0;
memset(out, 0, sizeof(*out));
@@ -162,7 +238,11 @@ int Node_ConnListenEndpoint(const tcp_connection_t* conn, struct sockaddr_storag
return 0;
}
size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpoints, size_t maxOut) {
uint64_t Node_ConnPeerNodeId(const tcp_connection_t* conn) {
return conn ? conn->peerNodeId : 0;
}
size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpoints, uint64_t* outNodeIds, size_t maxOut) {
if (!node || !outEndpoints || maxOut == 0) return 0;
size_t count = 0;
@@ -170,14 +250,16 @@ size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpo
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS && count < maxOut; ++i) {
tcp_connection_t* c = node->outboundClients[i].connection;
if (!c) continue;
if (!c || TcpConnection_IsDisconnectNotified(c)) continue; // ignore connections that are tearing down
struct sockaddr_storage ep;
if (!Node_ConnListenEndpoint(c, &ep)) continue;
int dup = 0;
for (size_t k = 0; k < count; ++k) {
if (NetNode_EndpointEqual(&outEndpoints[k], &ep)) { dup = 1; break; }
}
if (!dup) outEndpoints[count++] = ep;
if (dup) continue;
if (outNodeIds) outNodeIds[count] = c->peerNodeId;
outEndpoints[count++] = ep;
}
pthread_mutex_unlock(&node->outboundLock);
@@ -186,14 +268,16 @@ size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpo
pthread_mutex_lock(&node->server->clientsMutex);
for (size_t i = 0; i < node->server->maxClients && count < maxOut; ++i) {
tcp_connection_t* c = node->server->clientsArrPtr ? node->server->clientsArrPtr[i] : NULL;
if (!c) continue;
if (!c || TcpConnection_IsDisconnectNotified(c)) continue; // ignore connections that are tearing down
struct sockaddr_storage ep;
if (!Node_ConnListenEndpoint(c, &ep)) continue;
int dup = 0;
for (size_t k = 0; k < count; ++k) {
if (NetNode_EndpointEqual(&outEndpoints[k], &ep)) { dup = 1; break; }
}
if (!dup) outEndpoints[count++] = ep;
if (dup) continue;
if (outNodeIds) outNodeIds[count] = c->peerNodeId;
outEndpoints[count++] = ep;
}
pthread_mutex_unlock(&node->server->clientsMutex);
}
@@ -201,6 +285,44 @@ size_t Node_GetPeerEndpoints(net_node_t* node, struct sockaddr_storage* outEndpo
return count;
}
// Outcome of the identity check run once a connection's HELLO/ACK_HELLO has been parsed.
typedef enum {
NODE_IDENTITY_OK = 0,
NODE_IDENTITY_SELF, // the peer is this very node, reached through one of its own addresses
NODE_IDENTITY_DUPLICATE // we already hold a connection of this role to that node
} node_identity_result_t;
// Records the identity a peer advertised and decides whether the connection should survive.
// `conn->peerNodeId` and `conn->peerListenPort` must already be set from the handshake.
static node_identity_result_t Node_CheckPeerIdentity(net_node_t* node, tcp_connection_t* conn) {
if (!node || !conn || conn->peerNodeId == 0) return NODE_IDENTITY_OK; // peer too old to advertise one
struct sockaddr_storage ep;
int haveEp = Node_ConnListenEndpoint(conn, &ep);
if (conn->peerNodeId == localNodeId) {
// We dialled ourselves (or accepted our own dial). Remember the endpoint as our own so
// discovery stops offering it back to us, and drop the connection.
if (haveEp && node->discovery) {
NodeDiscovery_MarkSelfEndpoint(node->discovery, &ep);
}
return NODE_IDENTITY_SELF;
}
// Record the identity behind this endpoint even when the connection is about to be dropped as a
// duplicate: that is what lets discovery skip the peer's other addresses while we are connected
// to it, instead of dialling each of them in turn.
if (haveEp && node->discovery) {
NodeDiscovery_NoteIdentity(node->discovery, &ep, conn->peerNodeId);
}
if (Node_HasOtherConnectionToNode(node, conn, conn->peerNodeId)) {
return NODE_IDENTITY_DUPLICATE;
}
return NODE_IDENTITY_OK;
}
// Thunks routing UDP ping/pong events into the discovery state.
static void Node_OnPongThunk(udp_node_t* udp, const struct sockaddr_storage* from,
uint64_t nonce, int protoVersion, uint64_t rttMs, void* user) {
@@ -223,9 +345,106 @@ static void Node_OnPingTimeoutThunk(udp_node_t* udp, const struct sockaddr_stora
typedef enum {
NODE_BLOCK_REJECTED = 0,
NODE_BLOCK_ORPHAN_QUEUED = 1,
NODE_BLOCK_ACCEPTED = 2
NODE_BLOCK_ACCEPTED = 2,
NODE_BLOCK_DUPLICATE = 3 // already on our chain; not a fault, do not log it as a rejection
} node_block_accept_result_t;
// Delivery receipts for windowed sync -- see the contract in net_node.h. Written from peer io
// threads, drained by the REPL thread running `sync`, so it needs its own lock; it never calls back
// into chain.c or takes any other lock, so it cannot participate in a cycle.
#define NODE_DELIVERY_SLOTS 512
typedef struct {
uint64_t height;
node_delivery_status_t status;
bool valid;
} node_delivery_t;
static node_delivery_t g_deliveries[NODE_DELIVERY_SLOTS];
static size_t g_deliveryNext = 0;
static pthread_mutex_t g_deliveryLock = PTHREAD_MUTEX_INITIALIZER;
void Node_NoteBlockDelivered(uint64_t height, node_delivery_status_t status) {
pthread_mutex_lock(&g_deliveryLock);
// Refresh an existing receipt rather than adding a second one for the same height: a retried
// request would otherwise leave a stale receipt that the next window could consume by mistake.
for (size_t i = 0; i < NODE_DELIVERY_SLOTS; ++i) {
if (g_deliveries[i].valid && g_deliveries[i].height == height) {
g_deliveries[i].status = status;
pthread_mutex_unlock(&g_deliveryLock);
return;
}
}
g_deliveries[g_deliveryNext].height = height;
g_deliveries[g_deliveryNext].status = status;
g_deliveries[g_deliveryNext].valid = true;
g_deliveryNext = (g_deliveryNext + 1u) % NODE_DELIVERY_SLOTS;
pthread_mutex_unlock(&g_deliveryLock);
}
bool Node_TakeBlockDelivery(uint64_t height, node_delivery_status_t* outStatus) {
bool found = false;
pthread_mutex_lock(&g_deliveryLock);
for (size_t i = 0; i < NODE_DELIVERY_SLOTS; ++i) {
if (g_deliveries[i].valid && g_deliveries[i].height == height) {
if (outStatus) {
*outStatus = g_deliveries[i].status;
}
g_deliveries[i].valid = false; // consumed
found = true;
break;
}
}
pthread_mutex_unlock(&g_deliveryLock);
return found;
}
void Node_ResetBlockDeliveries(void) {
pthread_mutex_lock(&g_deliveryLock);
memset(g_deliveries, 0, sizeof(g_deliveries));
g_deliveryNext = 0;
pthread_mutex_unlock(&g_deliveryLock);
}
// Reclaims outbound slots whose peer has disconnected. Mirrors the inbound self-reclaim in
// TcpServer_clientthreadprocess: detach dead connections from their slots under outboundLock, then
// join their io threads and destroy/free them outside the lock. Pinned connections (a raw pointer
// is still held elsewhere, e.g. by an in-progress sync) are skipped and retried on a later tick.
static void Node_ReapDeadOutbound(net_node_t* node) {
if (!node) return;
tcp_connection_t* dead[MAX_CONS];
size_t deadCount = 0;
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS; ++i) {
tcp_connection_t* c = node->outboundClients[i].connection;
if (!c) continue;
if (!TcpConnection_IsDisconnectNotified(c)) continue; // still live
if (atomic_load(&c->pinCount) != 0) continue; // someone holds a raw pointer; retry later
// Detach the dead connection from its slot and reset the slot to a clean free state.
node->outboundClients[i].connection = NULL;
node->outboundClients[i].peerBlockHeight = 0;
dead[deadCount++] = c;
}
pthread_mutex_unlock(&node->outboundLock);
// Join + destroy outside the lock: the io thread's on_disconnect callback itself takes
// outboundLock, so joining under it would deadlock.
for (size_t i = 0; i < deadCount; ++i) {
tcp_connection_t* c = dead[i];
if (!pthread_equal(c->ioThread, pthread_self())) {
pthread_join(c->ioThread, NULL);
}
TcpConnection_Destroy(c);
free(c);
}
}
static void* Node_MaintenanceThread(void* arg) {
net_node_t* n = (net_node_t*)arg;
if (!n) return NULL;
@@ -238,6 +457,16 @@ static void* Node_MaintenanceThread(void* arg) {
BalanceSheet_SaveToFile(chainDataDir);
}
}
// Drop transactions too old to be worth holding, so the pool is not inflated by junk that
// will never be mined. Policy only -- a block containing one is still accepted.
{
const size_t pruned = TxMempool_PruneExpired(get_current_time_ms());
if (pruned > 0) {
printf("Maintenance: pruned %zu expired transaction(s) from the mempool\n", pruned);
}
}
// Reclaim outbound slots whose peer has disconnected so they can be reused.
Node_ReapDeadOutbound(n);
// Peer discovery tick: ping/query connected peers and connect to the best-ping discoveries.
if (n->discovery) {
NodeDiscovery_Iterate(n->discovery);
@@ -303,14 +532,8 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
}
}
// Validate block
if (!Block_IsFullyValid(blk)) {
printf("Rejected BLOCK_DATA at height %" PRIu64 " during validation\n", blockHeight);
DynArr_destroy(blk->transactions);
free(blk);
return NODE_BLOCK_REJECTED;
}
// The chain check has to come first now: PoW validity is chain-relative (the epoch DAG size and
// seed are derived from it), so there is nothing to validate against without a chain.
if (!currentChain) {
printf("Rejected BLOCK_DATA at height %" PRIu64 ": no active chain\n", blockHeight);
DynArr_destroy(blk->transactions);
@@ -318,33 +541,69 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
return NODE_BLOCK_REJECTED;
}
// Only the self-contained checks run here. Proof of work is verified by Chain_AddBlock, at the
// point a block actually joins the chain.
//
// PoW cannot be judged here because it is relative to the branch the block belongs to: the
// epoch seed is the last block of the previous epoch on ITS branch. For a block on a competing
// branch our chain gives the WRONG seed whenever the two diverge before that boundary, so
// checking it here rejected perfectly valid blocks and made any fork spanning an epoch boundary
// impossible to assemble. Deferring costs at most a slot in a pool that is already capped.
if (!Block_HasValidStructure(blk)) {
printf("Rejected BLOCK_DATA at height %" PRIu64 " during validation\n", blockHeight);
DynArr_destroy(blk->transactions);
free(blk);
return NODE_BLOCK_REJECTED;
}
// The orphan pool stamps the local tip height at first sight; that stamp drives the reorg
// penalty and must be taken now, not re-derived later from a moved tip.
uint64_t chainSize = Chain_Size(currentChain);
const uint64_t observedAtTipHeight = chainSize > 0 ? (chainSize - 1) : 0ULL;
// Temporary debug mode: force network-received blocks through the orphan pool to exercise reorg handling.
if (forceOrphanReorgEnabled && blk->header.blockNumber > 0) {
OrphanPool_Insert(blk, blockHeight);
OrphanPool_Insert(blk, blockHeight, observedAtTipHeight);
printf("Forced orphan BLOCK_DATA at height %" PRIu64 "\n", blockHeight);
return NODE_BLOCK_ORPHAN_QUEUED;
}
// If parent is missing, insert into orphan pool instead of rejecting immediately.
uint64_t chainSize = Chain_Size(currentChain);
if (blk->header.blockNumber > chainSize) {
// Parent(s) missing; queue as orphan
OrphanPool_Insert(blk, blockHeight);
OrphanPool_Insert(blk, blockHeight, observedAtTipHeight);
printf("Queued orphan BLOCK_DATA at height %" PRIu64 "\n", blockHeight);
return NODE_BLOCK_ORPHAN_QUEUED;
} else if (blk->header.blockNumber < chainSize) {
// Older block than current chain tip: reject
printf("Rejected BLOCK_DATA at height %" PRIu64 ": older than current chain\n", blockHeight);
// A block below our tip is either one we already have, or the lower half of a competing
// branch. Dropping both (as this used to) made any fork that diverges below the tip
// impossible to discover: the fork point itself was always thrown away.
block_t* local = NULL;
if (Chain_GetBlockCopy(currentChain, (size_t)blk->header.blockNumber, &local) && local) {
uint8_t localHash[32];
uint8_t incomingHash[32];
Block_CalculateHash(local, localHash);
Block_CalculateHash(blk, incomingHash);
Block_Destroy(local);
if (memcmp(localHash, incomingHash, 32) == 0) {
// Exactly the block we already have.
DynArr_destroy(blk->transactions);
free(blk);
return NODE_BLOCK_REJECTED;
return NODE_BLOCK_DUPLICATE;
}
}
OrphanPool_Insert(blk, blockHeight, observedAtTipHeight);
printf("Queued forked BLOCK_DATA at height %" PRIu64 " (below our tip) as orphan\n", blockHeight);
return NODE_BLOCK_ORPHAN_QUEUED;
} else {
// blk->header.blockNumber == chainSize -> candidate to append. Ensure prevHash matches current tip.
if (chainSize > 0) {
block_t* last = NULL;
if (!Chain_GetBlockCopy(currentChain, (size_t)(chainSize - 1), &last) || !last) {
// Can't verify parent; queue as orphan conservatively
OrphanPool_Insert(blk, blockHeight);
OrphanPool_Insert(blk, blockHeight, observedAtTipHeight);
printf("Queued orphan BLOCK_DATA at height %" PRIu64 " (unable to verify parent)\n", blockHeight);
if (last) Block_Destroy(last);
return NODE_BLOCK_ORPHAN_QUEUED;
@@ -353,7 +612,7 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
Block_CalculateHash(last, lastHash);
if (memcmp(lastHash, blk->header.prevHash, 32) != 0) {
// Conflicting block at same height; queue as orphan until resolved by a subsequent extension.
OrphanPool_Insert(blk, blockHeight);
OrphanPool_Insert(blk, blockHeight, observedAtTipHeight);
Block_Destroy(last);
printf("Queued conflicting BLOCK_DATA at same height %" PRIu64 " as orphan\n", blockHeight);
return NODE_BLOCK_ORPHAN_QUEUED;
@@ -363,28 +622,16 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
}
if (!Chain_AddBlock(currentChain, blk)) {
// Chain_AddBlock failed; cleanup
// Chain_AddBlock failed; cleanup. Safe either way: if it failed before taking the block we
// still own the transactions, and if it failed after (the ledger pass can fail with the
// block already pushed) our pointer to them was cleared, so this frees only the wrapper.
printf("Rejected BLOCK_DATA at height %" PRIu64 " during chain add\n", blockHeight);
if (blk->transactions) {
DynArr_destroy(blk->transactions);
}
free(blk);
Block_Destroy(blk);
return NODE_BLOCK_REJECTED;
}
uint64_t coinbaseAmount = 0;
if (blk->transactions) {
for (size_t i = 0; i < DynArr_size(blk->transactions); ++i) {
signed_transaction_t* tx = (signed_transaction_t*)DynArr_at(blk->transactions, i);
if (tx && Address_IsCoinbase(tx->transaction.senderAddress)) {
coinbaseAmount = tx->transaction.amount1;
break;
}
}
}
(void)uint256_add_u64(&currentSupply, coinbaseAmount);
currentReward = CalculateBlockReward(currentSupply, currentChain);
// currentSupply/currentReward are advanced inside Chain_AddBlock, so that every path that
// appends (mining, this one, orphan attach, reorg) keeps them consistent.
// Persist on accept if requested
if (persist) {
@@ -392,8 +639,9 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
BalanceSheet_SaveToFile(chainDataDir);
}
// Chain_AddBlock copied the block into the chain; free our temporary wrapper but do NOT destroy transactions (they are freed by Chain_SaveToFile when persisted)
free(blk);
// Chain_AddBlock took ownership of the transaction array and cleared our pointer to it, so
// destroying the wrapper here frees only the wrapper.
Block_Destroy(blk);
// Attempt to attach any orphans that may now have their parents present.
size_t attached = OrphanPool_AttemptAttach(currentChain);
if (attached > 0) {
@@ -496,22 +744,57 @@ void Node_Destroy(net_node_t* node) {
return;
}
// Stop the maintenance thread first: it runs the outbound reaper (which touches outboundClients
// and outboundLock) and the discovery tick, so it must not run concurrently with the teardown
// below or against soon-to-be-destroyed state.
if (node->maintenanceRunning) {
node->maintenanceRunning = 0;
pthread_join(node->maintenanceThread, NULL);
}
// Detach every outbound connection from its slot under outboundLock, then tear the connections
// down outside it -- the same pattern Node_ReapDeadOutbound uses, and for the same two reasons.
//
// Calling TcpClient_Destroy directly here instead raced with still-running inbound client
// threads: those read outboundClients[i].connection under outboundLock (via
// Node_HasLiveConnectionTo), while TcpClient_Disconnect cleared the same field with no lock
// held. The lock cannot simply be held across the destroy, because that path joins the io
// thread whose on_disconnect callback takes outboundLock itself.
tcp_connection_t* outbound[MAX_CONS];
size_t outboundToClose = 0;
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS; ++i) {
TcpClient_Destroy(&node->outboundClients[i]);
tcp_connection_t* conn = node->outboundClients[i].connection;
if (!conn) continue;
node->outboundClients[i].connection = NULL;
node->outboundClients[i].peerBlockHeight = 0;
outbound[outboundToClose++] = conn;
}
node->outboundCount = 0;
pthread_mutex_unlock(&node->outboundLock);
for (size_t i = 0; i < outboundToClose; ++i) {
tcp_connection_t* conn = outbound[i];
TcpConnection_RequestClose(conn);
if (!pthread_equal(conn->ioThread, pthread_self())) {
pthread_join(conn->ioThread, NULL);
}
if (!TcpConnection_IsDisconnectNotified(conn) && conn->on_disconnect) {
TcpConnection_MarkDisconnectNotified(conn);
conn->on_disconnect(conn);
}
TcpConnection_Destroy(conn);
free(conn);
}
if (node->server) {
TcpServer_Stop(node->server);
TcpServer_Destroy(node->server);
}
// Stop maintenance thread (no more discovery ticks after this)
if (node->maintenanceRunning) {
node->maintenanceRunning = 0;
pthread_join(node->maintenanceThread, NULL);
}
// Tear down UDP + discovery. Stop UDP first so no pong/timeout callback races the destroy.
if (node->udpNode) {
UdpNode_Stop(node->udpNode);
@@ -561,11 +844,18 @@ int Node_ConnectPeer(net_node_t* node, const char* ip, unsigned short port) {
return -1;
}
// Never dial ourselves. Without this an echo-back (or a gossiped copy of one of our own
// addresses) can chain into a self-connection per maintenance tick until the slots run out.
struct sockaddr_storage target;
int haveTarget = NetNode_MakeEndpoint(ip, port, &target);
if (haveTarget && node->discovery && NodeDiscovery_IsSelfEndpoint(node->discovery, &target)) {
return -1;
}
// Enforce a single outbound connection per endpoint: if we already have an outbound to this
// (ip, port), do not open a second one. (Inbound from the same endpoint is still allowed - that
// is the peer's own outbound to us.)
struct sockaddr_storage target;
if (NetNode_MakeEndpoint(ip, port, &target) && Node_HasOutboundTo(node, &target)) {
if (haveTarget && Node_HasOutboundTo(node, &target)) {
return 0; // already connected outbound to this endpoint
}
@@ -727,27 +1017,23 @@ void Node_Server_OnData(tcp_connection_t* client) {
client->peerListenPort = peerListenPort;
}
printf("Received HELLO from node %u: protoVersion=%u, blockHeight=%" PRIu64 ", listenPort=%u\n",
// Optional trailing node identity, same length-guarded deal.
if (client && payloadLen >= sizeof(uint32_t) + sizeof(uint64_t) + sizeof(uint16_t) + sizeof(uint64_t)) {
uint64_t peerNodeId;
memcpy(&peerNodeId, payload + sizeof(protoVersion) + sizeof(blockHeight) + sizeof(uint16_t), sizeof(peerNodeId));
client->peerNodeId = peerNodeId;
}
printf("Received HELLO from node %u: protoVersion=%u, blockHeight=%" PRIu64 ", listenPort=%u, nodeId=%016" PRIx64 "\n",
client ? client->connectionId : 0U, protoVersion, blockHeight,
client ? client->peerListenPort : 0U);
client ? client->peerListenPort : 0U, client ? client->peerNodeId : 0ULL);
// Enforce a single inbound connection per endpoint. Now that we know this peer's listen
// port, drop this connection if another inbound from the same endpoint already exists
// (keep the established one). An outbound to the same endpoint is unaffected - that is
// this node's own connection to the peer.
if (client && client->peerListenPort != 0) {
net_node_t* dupNode = Node_FromConnection(client);
struct sockaddr_storage myEp;
if (dupNode && Node_ConnListenEndpoint(client, &myEp) &&
Node_HasOtherInboundFrom(dupNode, client, &myEp)) {
printf("Rejecting duplicate inbound connection %u (already have an inbound from this endpoint)\n",
client->connectionId);
TcpConnection_RequestClose(client);
return;
}
}
// Craft and send ACK_HELLO (echo protoVersion, our height, and our own listen port)
// Craft and send ACK_HELLO (echo protoVersion, our height, our own listen port and our
// identity). This goes out before any decision to drop the connection: the ACK is what
// tells the dialer whose address it just reached, so an endpoint that turns out to be
// another address of a peer it already talks to (or one of its own) is recognised as
// such instead of being redialled forever. shutdown() flushes what is already queued,
// so the peer still receives this even though we close immediately after.
uint8_t ackBuf[100];
uint8_t* ackData = ackBuf;
size_t ackOffset = 0;
@@ -759,9 +1045,47 @@ void Node_Server_OnData(tcp_connection_t* client) {
uint16_t myListenPort = (uint16_t)listenPort;
memcpy(ackData + ackOffset, &myListenPort, sizeof(myListenPort));
ackOffset += sizeof(myListenPort);
uint64_t myNodeId = localNodeId;
memcpy(ackData + ackOffset, &myNodeId, sizeof(myNodeId));
ackOffset += sizeof(myNodeId);
Node_SendPacket(Node_FromConnection(client), client, PACKET_TYPE_ACK_HELLO, ackData, ackOffset);
// Enforce one connection per node, identified by the advertised nodeId rather than by
// the address it happens to reach us from.
if (client) {
net_node_t* idNode = Node_FromConnection(client);
node_identity_result_t identity = Node_CheckPeerIdentity(idNode, client);
if (identity == NODE_IDENTITY_SELF) {
printf("Rejecting inbound connection %u: it is this node talking to itself\n",
client->connectionId);
TcpConnection_RequestClose(client);
return;
}
if (identity == NODE_IDENTITY_DUPLICATE) {
printf("Rejecting duplicate inbound connection %u (already connected to node %016" PRIx64 ")\n",
client->connectionId, client->peerNodeId);
TcpConnection_RequestClose(client);
return;
}
}
// Endpoint-level fallback for peers that advertise no identity: drop this connection if
// another inbound from the same endpoint already exists (keep the established one). An
// outbound to the same endpoint is unaffected - that is this node's own connection to
// the peer.
if (client && client->peerNodeId == 0 && client->peerListenPort != 0) {
net_node_t* dupNode = Node_FromConnection(client);
struct sockaddr_storage myEp;
if (dupNode && Node_ConnListenEndpoint(client, &myEp) &&
Node_HasOtherInboundFrom(dupNode, client, &myEp)) {
printf("Rejecting duplicate inbound connection %u (already have an inbound from this endpoint)\n",
client->connectionId);
TcpConnection_RequestClose(client);
return;
}
}
break;
}
case PACKET_TYPE_ACK_HELLO: {
@@ -893,6 +1217,8 @@ void Node_Server_OnData(tcp_connection_t* client) {
}
} else if (result == NODE_BLOCK_ORPHAN_QUEUED) {
printf("Queued orphan BROADCAST_BLOCK from node %u\n", client ? client->connectionId : 0U);
} else if (result == NODE_BLOCK_DUPLICATE) {
// Already on our chain (a peer relayed it to us twice); not an error.
} else {
printf("Rejected BROADCAST_BLOCK from node %u\n", client ? client->connectionId : 0U);
}
@@ -917,7 +1243,14 @@ void Node_Server_OnData(tcp_connection_t* client) {
return;
}
// Push to mempool if it's not already present
// Push to mempool if it's not already present, subject to admission policy.
// Policy only: a block containing this transaction is still accepted even if we
// decline to hold or relay it ourselves.
if (!TxMempool_PolicyAccepts(&tx, get_current_time_ms())) {
printf("Declined transaction from node %u: timestamp outside the accepted window\n",
client ? client->connectionId : 0U);
return;
}
if (!TxMempool_Lookup(txHash, &tx)) {
if (TxMempool_Insert(tx) >= 0) {
printf("Added transaction %s from node %u to mempool\n", txHashHex, client ? client->connectionId : 0U);
@@ -983,6 +1316,7 @@ void Node_Server_OnDisconnect(tcp_connection_t* client) {
net_node_t* node = Node_FromConnection(client);
Node_ForwardDisconnect(node, client);
printf("Inbound node disconnected: %u\n", client ? client->connectionId : 0U);
Node_HandlePeerDisconnect(node, client);
}
void Node_Client_OnConnect(tcp_connection_t* client) {
@@ -1006,6 +1340,10 @@ void Node_Client_OnConnect(tcp_connection_t* client) {
uint16_t myListenPort = (uint16_t)listenPort;
memcpy((unsigned char*)data + offset, &myListenPort, sizeof(myListenPort));
offset += sizeof(myListenPort);
// ...and who we are, so the peer can tell this connection apart from our other addresses
uint64_t myNodeId = localNodeId;
memcpy((unsigned char*)data + offset, &myNodeId, sizeof(myNodeId));
offset += sizeof(myNodeId);
Node_SendPacket(node, client, PACKET_TYPE_HELLO, data, offset);
}
@@ -1050,10 +1388,36 @@ void Node_Client_OnData(tcp_connection_t* client) {
client->peerListenPort = peerListenPort;
}
printf("Received ACK_HELLO from node %u with protoVersion %u and blockHeight %" PRIu64 "\n", client ? client->connectionId : 0U, protoVersion, blockHeight);
// Optional trailing node identity, same length-guarded deal.
if (client && payloadLen >= sizeof(uint32_t) + sizeof(uint64_t) + sizeof(uint16_t) + sizeof(uint64_t)) {
uint64_t peerNodeId;
memcpy(&peerNodeId, payload + sizeof(protoVersion) + sizeof(blockHeight) + sizeof(uint16_t), sizeof(peerNodeId));
client->peerNodeId = peerNodeId;
}
printf("Received ACK_HELLO from node %u with protoVersion %u, blockHeight %" PRIu64 " and nodeId %016" PRIx64 "\n",
client ? client->connectionId : 0U, protoVersion, blockHeight, client ? client->peerNodeId : 0ULL);
// Store peer-advertised height on matching outbound client
net_node_t* node = Node_FromConnection(client);
// The dialed endpoint may well be one of our own addresses, or another address of a
// peer we already talk to - neither is worth a connection.
if (client) {
node_identity_result_t identity = Node_CheckPeerIdentity(node, client);
if (identity == NODE_IDENTITY_SELF) {
printf("Closing outbound connection %u: it loops back to this node\n", client->connectionId);
TcpConnection_RequestClose(client);
return;
}
if (identity == NODE_IDENTITY_DUPLICATE) {
printf("Closing outbound connection %u: already connected to node %016" PRIx64 " on another address\n",
client->connectionId, client->peerNodeId);
TcpConnection_RequestClose(client);
return;
}
}
if (node) {
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS; ++i) {
@@ -1081,6 +1445,19 @@ void Node_Client_OnData(tcp_connection_t* client) {
uint64_t blockHeight = 0;
memcpy(&blockHeight, payload, sizeof(blockHeight));
node_block_accept_result_t result = Node_ParseAndAcceptBlock(payload, payloadLen, true);
// Receipt for the windowed sync. BLOCK_DATA is only ever sent in reply to a
// FETCH_BLOCK, so recording it here (and not for BROADCAST_BLOCK) tells the sync
// loop the peer answered, whether or not the block could join our chain.
node_delivery_status_t deliveryStatus = NODE_DELIVERY_REJECTED;
switch (result) {
case NODE_BLOCK_ACCEPTED: deliveryStatus = NODE_DELIVERY_APPENDED; break;
case NODE_BLOCK_DUPLICATE: deliveryStatus = NODE_DELIVERY_DUPLICATE; break;
case NODE_BLOCK_ORPHAN_QUEUED: deliveryStatus = NODE_DELIVERY_ORPHANED; break;
default: deliveryStatus = NODE_DELIVERY_REJECTED; break;
}
Node_NoteBlockDelivered(blockHeight, deliveryStatus);
if (result == NODE_BLOCK_ACCEPTED) {
printf("Accepted BLOCK_DATA from node %u\n", client ? client->connectionId : 0U);
net_node_t* node = Node_FromConnection(client);
@@ -1101,6 +1478,8 @@ void Node_Client_OnData(tcp_connection_t* client) {
}
} else if (result == NODE_BLOCK_ORPHAN_QUEUED) {
printf("Queued orphan BLOCK_DATA from node %u\n", client ? client->connectionId : 0U);
} else if (result == NODE_BLOCK_DUPLICATE) {
// Already on our chain (a peer relayed it to us twice); not an error.
} else {
printf("Rejected BLOCK_DATA from node %u\n", client ? client->connectionId : 0U);
}
@@ -1132,6 +1511,8 @@ void Node_Client_OnData(tcp_connection_t* client) {
}
} else if (result == NODE_BLOCK_ORPHAN_QUEUED) {
printf("Queued orphan BROADCAST_BLOCK from node %u\n", client ? client->connectionId : 0U);
} else if (result == NODE_BLOCK_DUPLICATE) {
// Already on our chain (a peer relayed it to us twice); not an error.
} else {
printf("Rejected BROADCAST_BLOCK from node %u\n", client ? client->connectionId : 0U);
}
@@ -1202,6 +1583,7 @@ void Node_Client_OnDisconnect(tcp_connection_t* client) {
Node_ForwardDisconnect(node, client);
printf("Outbound node disconnected: %u\n", client ? client->connectionId : 0U);
Node_HandlePeerDisconnect(node, client);
}
int Node_GetBestOutboundPeer(net_node_t* node, tcp_connection_t** outConn, uint64_t* outHeight) {
@@ -1212,13 +1594,16 @@ int Node_GetBestOutboundPeer(net_node_t* node, tcp_connection_t** outConn, uint6
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS; ++i) {
if (node->outboundClients[i].connection) {
if (node->outboundClients[i].peerBlockHeight > bestH || best == NULL) {
best = node->outboundClients[i].connection;
tcp_connection_t* c = node->outboundClients[i].connection;
if (!c || TcpConnection_IsDisconnectNotified(c)) continue; // don't hand out a dead peer
if (best == NULL || node->outboundClients[i].peerBlockHeight > bestH) {
best = c;
bestH = node->outboundClients[i].peerBlockHeight;
}
}
}
// Pin the winner while still holding outboundLock so the reaper cannot free it out from under
// the caller (which uses the raw pointer after this lock is released). Caller must Unpin.
if (best) TcpConnection_Pin(best);
pthread_mutex_unlock(&node->outboundLock);
if (!best) return -1;
@@ -1255,13 +1640,13 @@ void Node_BroadcastChainRange(net_node_t* node, size_t startHeightInclusive, tcp
unsigned char hash[32];
Block_CalculateHash(blk, hash);
// Dedupe using seenBlocks
// Dedupe using seenBlocks. The hash is only recorded once the block has actually gone out
// to at least one peer: marking it here unconditionally meant that a block relayed while no
// peer was connected (or while every peer was filtered out below) was never offered again.
int seen = 0;
pthread_mutex_lock(&node->seenLock);
if (DynSet_Contains(node->seenBlocks, hash)) {
seen = 1;
} else {
DynSet_Insert(node->seenBlocks, hash);
}
pthread_mutex_unlock(&node->seenLock);
@@ -1289,17 +1674,87 @@ void Node_BroadcastChainRange(net_node_t* node, size_t startHeightInclusive, tcp
memcpy(payload + off, tx, sizeof(signed_transaction_t)); off += sizeof(signed_transaction_t);
}
// Snapshot outbound clients and send
// Collect one connection per distinct peer, then send with no lock held.
//
// A peer we both dialled and were dialled by occupies two connections (one outbound, one
// inbound). Sending on both delivers every block twice, and the receiver logs the second
// copy as a rejection. Peers are identified by peerNodeId rather than by endpoint, because
// a multi-homed host reaches us from several addresses and an inbound connection carries an
// ephemeral port while the outbound one carries the listen port.
//
// Sends happen outside outboundLock/clientsMutex on purpose: Node_SendPacket writes to a
// socket and can block when the peer is slow to read, and holding the server's clientsMutex
// across that stalls the accept path and every other user of it.
tcp_connection_t* targets[MAX_CONS * 2];
uint64_t targetNodeIds[MAX_CONS * 2];
size_t targetCount = 0;
uint64_t sourceNodeId = sourceConn ? sourceConn->peerNodeId : 0ULL;
// Skip a connection if it is the source, belongs to the source's node, or duplicates a peer
// we have already queued.
#define NODE_RELAY_SHOULD_SKIP(conn) ( \
(conn) == sourceConn || \
((sourceNodeId != 0ULL) && ((conn)->peerNodeId == sourceNodeId)) || \
(sourceConn && (sourceNodeId == 0ULL) && TcpConnection_PeerAddrEqual((conn), sourceConn)))
pthread_mutex_lock(&node->outboundLock);
for (size_t i = 0; i < MAX_CONS; ++i) {
for (size_t i = 0; i < MAX_CONS && targetCount < (MAX_CONS * 2); ++i) {
tcp_connection_t* conn = node->outboundClients[i].connection;
if (!conn) continue;
if (conn == sourceConn) continue;
if (sourceConn && TcpConnection_PeerAddrEqual(conn, sourceConn)) continue;
Node_SendPacket(node, conn, PACKET_TYPE_BROADCAST_BLOCK, payload, off);
if (!conn || TcpConnection_IsDisconnectNotified(conn)) continue;
if (NODE_RELAY_SHOULD_SKIP(conn)) continue;
bool duplicate = false;
for (size_t t = 0; t < targetCount; ++t) {
if (conn->peerNodeId != 0ULL && targetNodeIds[t] == conn->peerNodeId) { duplicate = true; break; }
}
if (duplicate) continue;
TcpConnection_Pin(conn);
targetNodeIds[targetCount] = conn->peerNodeId;
targets[targetCount++] = conn;
}
pthread_mutex_unlock(&node->outboundLock);
// Inbound peers too. Broadcasting only to outbound connections meant that in a two-node
// setup the node that was dialled never pushed anything back, and the dialer only ever
// learned about new blocks through a manual `sync`.
if (node->server) {
pthread_mutex_lock(&node->server->clientsMutex);
for (size_t i = 0; i < node->server->maxClients && targetCount < (MAX_CONS * 2); ++i) {
tcp_connection_t* conn = node->server->clientsArrPtr ? node->server->clientsArrPtr[i] : NULL;
if (!conn || TcpConnection_IsDisconnectNotified(conn)) continue;
if (NODE_RELAY_SHOULD_SKIP(conn)) continue;
bool duplicate = false;
for (size_t t = 0; t < targetCount; ++t) {
if (conn->peerNodeId != 0ULL && targetNodeIds[t] == conn->peerNodeId) { duplicate = true; break; }
}
if (duplicate) continue;
TcpConnection_Pin(conn);
targetNodeIds[targetCount] = conn->peerNodeId;
targets[targetCount++] = conn;
}
pthread_mutex_unlock(&node->server->clientsMutex);
}
#undef NODE_RELAY_SHOULD_SKIP
size_t delivered = 0;
for (size_t t = 0; t < targetCount; ++t) {
if (Node_SendPacket(node, targets[t], PACKET_TYPE_BROADCAST_BLOCK, payload, off) == 0) {
delivered++;
}
TcpConnection_Unpin(targets[t]);
}
if (delivered > 0) {
pthread_mutex_lock(&node->seenLock);
DynSet_Insert(node->seenBlocks, hash);
pthread_mutex_unlock(&node->seenLock);
}
free(payload);
Block_Destroy(blk);
}
@@ -1311,8 +1766,9 @@ void Node_GetClientList(net_node_t* node, tcp_connection_t** outClients, size_t*
pthread_mutex_lock(&node->outboundLock);
size_t count = 0;
for (size_t i = 0; i < MAX_CONS; ++i) {
if (node->outboundClients[i].connection) {
outClients[count++] = node->outboundClients[i].connection;
tcp_connection_t* c = node->outboundClients[i].connection;
if (c && !TcpConnection_IsDisconnectNotified(c)) { // skip connections that are tearing down
outClients[count++] = c;
}
}
pthread_mutex_unlock(&node->outboundLock);
src/nets/nodediscovery.c
+284 -11
View File
@@ -8,9 +8,12 @@
#include <netinet/in.h>
#include <sys/socket.h>
#include <ifaddrs.h>
#include <constants.h>
#include <dynarr.h>
#include <numgen.h>
#include <runtime_state.h>
#include <utils.h>
// Wire layout of a single peer endpoint inside a PEERS payload:
@@ -29,19 +32,29 @@ typedef enum {
typedef struct {
struct sockaddr_storage addr; // listen endpoint (port already set to the peer's listen port)
uint64_t pingMs; // measured UDP RTT, UINT64_MAX if unknown
uint64_t nodeId; // identity of the node behind this endpoint, 0 while unknown
uint32_t hop; // distance from us (0 = directly connected)
discovery_state_t state;
int pingPending; // 1 while a ping is outstanding (matched by address on pong/timeout).
// The UDP layer generates its own nonce, so we can't match by nonce here.
uint64_t lastPingMs; // when we last sent a ping
uint64_t lastQueryMs; // when we last sent GET_PEERS to it
uint64_t lastConnectMs; // when we last attempted a connect to it
} discovered_peer_t;
// When we last dialed an endpoint. Kept outside the peer table on purpose: a peer entry is struck
// the moment its connection drops, and if the dial history went with it, an endpoint that hangs up
// on us would be re-learned through gossip and redialed on every single tick.
typedef struct {
struct sockaddr_storage addr;
uint64_t lastMs;
} discovery_attempt_t;
struct node_discovery {
net_node_t* node;
udp_node_t* udpNode;
DynArr* peers; // of discovered_peer_t
DynArr* selfEndpoints; // of struct sockaddr_storage - our own listen endpoints
DynArr* connectAttempts; // of discovery_attempt_t
pthread_mutex_t lock;
};
@@ -64,6 +77,99 @@ static int Discovery_AddrEqual(const struct sockaddr_storage* a, const struct so
return 0;
}
// Rejects endpoints that can never be dialed as written. IPv6 in particular hands us plenty of
// these: link-local addresses are meaningless without the scope id (which the wire format does not
// carry), and the unspecified/multicast ranges are never a peer. Loopback stays allowed so several
// nodes can still be run on one machine on different ports.
static int Discovery_IsUsableAddr(const struct sockaddr_storage* addr) {
if (addr->ss_family == AF_INET) {
const struct sockaddr_in* a = (const struct sockaddr_in*)addr;
if (a->sin_port == 0) return 0;
uint32_t host = ntohl(a->sin_addr.s_addr);
if (host == INADDR_ANY || host == INADDR_BROADCAST) return 0;
if ((host >> 28) == 0xE) return 0; // 224.0.0.0/4 multicast
if ((host & 0xFFFF0000u) == 0xA9FE0000u) return 0; // 169.254.0.0/16 link-local
return 1;
}
if (addr->ss_family == AF_INET6) {
const struct sockaddr_in6* a = (const struct sockaddr_in6*)addr;
if (a->sin6_port == 0) return 0;
if (IN6_IS_ADDR_UNSPECIFIED(&a->sin6_addr)) return 0;
if (IN6_IS_ADDR_MULTICAST(&a->sin6_addr)) return 0;
if (IN6_IS_ADDR_LINKLOCAL(&a->sin6_addr)) return 0; // unusable without a scope id
if (IN6_IS_ADDR_SITELOCAL(&a->sin6_addr)) return 0; // deprecated fec0::/10
return 1;
}
return 0;
}
// Rewrites an IPv4-mapped IPv6 endpoint (::ffff:a.b.c.d) as plain IPv4, so the same host never
// occupies two entries. Matches the normalisation Node_ConnListenEndpoint does.
static void Discovery_NormaliseAddr(struct sockaddr_storage* addr) {
if (addr->ss_family != AF_INET6) return;
struct sockaddr_in6* a = (struct sockaddr_in6*)addr;
if (!IN6_IS_ADDR_V4MAPPED(&a->sin6_addr)) return;
struct in_addr v4;
memcpy(&v4, ((const uint8_t*)&a->sin6_addr) + 12, sizeof(v4));
uint16_t port = a->sin6_port;
memset(addr, 0, sizeof(*addr));
struct sockaddr_in* o = (struct sockaddr_in*)addr;
o->sin_family = AF_INET;
o->sin_addr = v4;
o->sin_port = port;
}
// Returns non-zero if addr is one of our own listen endpoints. Caller holds disc->lock.
static int Discovery_IsSelfUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = DynArr_size(disc->selfEndpoints);
for (size_t i = 0; i < n; ++i) {
const struct sockaddr_storage* self = (const struct sockaddr_storage*)DynArr_at(disc->selfEndpoints, i);
if (Discovery_AddrEqual(self, addr)) return 1;
}
return 0;
}
// Adds addr to the self set if not already there. Caller holds disc->lock.
static void Discovery_AddSelfUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
if (Discovery_IsSelfUnlocked(disc, addr)) return;
DynArr_push_back(disc->selfEndpoints, (void*)addr);
}
// Seeds the self set with (local interface address, our listen port) for every address this host
// carries. A multi-homed host - the normal case under IPv6, where a machine holds a global, a
// temporary privacy and a link-local address at once - is otherwise unable to tell its own
// endpoints from a peer's when they come back around through peer exchange.
static void Discovery_SeedSelfEndpoints(node_discovery_t* disc) {
struct ifaddrs* ifa = NULL;
if (getifaddrs(&ifa) != 0 || !ifa) return;
for (struct ifaddrs* it = ifa; it; it = it->ifa_next) {
if (!it->ifa_addr) continue;
struct sockaddr_storage ep;
memset(&ep, 0, sizeof(ep));
if (it->ifa_addr->sa_family == AF_INET) {
struct sockaddr_in* o = (struct sockaddr_in*)&ep;
memcpy(o, it->ifa_addr, sizeof(struct sockaddr_in));
o->sin_port = htons(listenPort);
} else if (it->ifa_addr->sa_family == AF_INET6) {
struct sockaddr_in6* o = (struct sockaddr_in6*)&ep;
memcpy(o, it->ifa_addr, sizeof(struct sockaddr_in6));
o->sin6_port = htons(listenPort);
o->sin6_scope_id = 0; // endpoints on the wire are scopeless; compare them the same way
} else {
continue;
}
Discovery_NormaliseAddr(&ep);
Discovery_AddSelfUnlocked(disc, &ep);
}
freeifaddrs(ifa);
}
static discovered_peer_t* Discovery_FindPtr(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = DynArr_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
@@ -73,9 +179,13 @@ static discovered_peer_t* Discovery_FindPtr(node_discovery_t* disc, const struct
return NULL;
}
// Insert addr if not already present. Returns a pointer to the (existing or new) entry, or NULL
// if the table is full. Note: the returned pointer is invalidated by any later push_back.
// Insert addr if not already present. Returns a pointer to the (existing or new) entry, or NULL if
// the address is unusable, is one of our own, or the table is full. Note: the returned pointer is
// invalidated by any later push_back.
static discovered_peer_t* Discovery_Upsert(node_discovery_t* disc, const struct sockaddr_storage* addr, uint32_t hop) {
if (!Discovery_IsUsableAddr(addr)) return NULL;
if (Discovery_IsSelfUnlocked(disc, addr)) return NULL; // never track, ping or dial ourselves
discovered_peer_t* existing = Discovery_FindPtr(disc, addr);
if (existing) {
if (hop < existing->hop) existing->hop = hop; // keep the shortest known distance
@@ -87,12 +197,71 @@ static discovered_peer_t* Discovery_Upsert(node_discovery_t* disc, const struct
memset(&np, 0, sizeof(np));
np.addr = *addr;
np.pingMs = UINT64_MAX;
np.nodeId = 0;
np.hop = hop;
np.state = DISCOVERY_STATE_NEW;
DynArr_push_back(disc->peers, &np);
return (discovered_peer_t*)DynArr_at(disc->peers, DynArr_size(disc->peers) - 1);
}
// Returns non-zero if addr may be dialed again, i.e. we have not tried it within the retry window.
// Caller holds disc->lock.
static int Discovery_ConnectCooledDown(node_discovery_t* disc, const struct sockaddr_storage* addr, uint64_t now) {
size_t n = DynArr_size(disc->connectAttempts);
for (size_t i = 0; i < n; ++i) {
const discovery_attempt_t* a = (const discovery_attempt_t*)DynArr_at(disc->connectAttempts, i);
if (Discovery_AddrEqual(&a->addr, addr)) {
return (now - a->lastMs) >= DISCOVERY_CONNECT_RETRY_MS;
}
}
return 1; // never dialed
}
// Stamps a dial attempt against addr, evicting the stalest record once the table is full.
// Caller holds disc->lock.
static void Discovery_NoteConnectAttempt(node_discovery_t* disc, const struct sockaddr_storage* addr, uint64_t now) {
size_t n = DynArr_size(disc->connectAttempts);
size_t oldestIdx = 0;
uint64_t oldestMs = UINT64_MAX;
for (size_t i = 0; i < n; ++i) {
discovery_attempt_t* a = (discovery_attempt_t*)DynArr_at(disc->connectAttempts, i);
if (Discovery_AddrEqual(&a->addr, addr)) {
a->lastMs = now;
return;
}
if (a->lastMs < oldestMs) {
oldestMs = a->lastMs;
oldestIdx = i;
}
}
if (n >= DISCOVERY_MAX_KNOWN_PEERS) {
discovery_attempt_t* victim = (discovery_attempt_t*)DynArr_at(disc->connectAttempts, oldestIdx);
victim->addr = *addr;
victim->lastMs = now;
return;
}
discovery_attempt_t na;
memset(&na, 0, sizeof(na));
na.addr = *addr;
na.lastMs = now;
DynArr_push_back(disc->connectAttempts, &na);
}
// Drops the entry for addr, if any. Caller holds disc->lock.
static void Discovery_RemoveUnlocked(node_discovery_t* disc, const struct sockaddr_storage* addr) {
size_t n = DynArr_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)DynArr_at(disc->peers, i);
if (Discovery_AddrEqual(&p->addr, addr)) {
DynArr_remove(disc->peers, i);
return;
}
}
}
static int Discovery_AddrToWire(const struct sockaddr_storage* addr, unsigned char out[DISCOVERY_WIRE_ENTRY_SIZE]) {
memset(out, 0, DISCOVERY_WIRE_ENTRY_SIZE);
if (addr->ss_family == AF_INET) {
@@ -131,6 +300,7 @@ static int Discovery_WireToAddr(const unsigned char in[DISCOVERY_WIRE_ENTRY_SIZE
a->sin6_family = AF_INET6;
memcpy(&a->sin6_addr, in + 1, sizeof(struct in6_addr));
a->sin6_port = htons(port);
Discovery_NormaliseAddr(out); // a v4-mapped sender must not become a second entry
return port != 0;
}
return 0;
@@ -166,13 +336,31 @@ node_discovery_t* NodeDiscovery_Create(net_node_t* node, udp_node_t* udpNode) {
free(disc);
return NULL;
}
disc->selfEndpoints = DYNARR_CREATE(struct sockaddr_storage, 8);
if (!disc->selfEndpoints) {
DynArr_destroy(disc->peers);
free(disc);
return NULL;
}
disc->connectAttempts = DYNARR_CREATE(discovery_attempt_t, 16);
if (!disc->connectAttempts) {
DynArr_destroy(disc->selfEndpoints);
DynArr_destroy(disc->peers);
free(disc);
return NULL;
}
pthread_mutex_init(&disc->lock, NULL);
// Nothing else is running yet, so the self set can be seeded without taking the lock.
Discovery_SeedSelfEndpoints(disc);
return disc;
}
void NodeDiscovery_Destroy(node_discovery_t* disc) {
if (!disc) return;
if (disc->peers) DynArr_destroy(disc->peers);
if (disc->selfEndpoints) DynArr_destroy(disc->selfEndpoints);
if (disc->connectAttempts) DynArr_destroy(disc->connectAttempts);
pthread_mutex_destroy(&disc->lock);
free(disc);
}
@@ -211,12 +399,14 @@ void NodeDiscovery_OnPingTimeout(node_discovery_t* disc, const struct sockaddr_s
void NodeDiscovery_OnGetPeers(node_discovery_t* disc, tcp_connection_t* fromConn) {
if (!disc || !fromConn) return;
// Snapshot our current peers' listen endpoints (inbound + outbound).
// Snapshot our current peers' listen endpoints (inbound + outbound) and their identities.
struct sockaddr_storage all[MAX_CONS * 2];
size_t total = Node_GetPeerEndpoints(disc->node, all, sizeof(all) / sizeof(all[0]));
uint64_t allIds[MAX_CONS * 2];
size_t total = Node_GetPeerEndpoints(disc->node, all, allIds, sizeof(all) / sizeof(all[0]));
struct sockaddr_storage reqEndpoint;
int haveReq = Node_ConnListenEndpoint(fromConn, &reqEndpoint);
uint64_t reqNodeId = Node_ConnPeerNodeId(fromConn);
// Build the response payload: [uint16 count][entries...], capped and sampled for spread.
unsigned char payload[sizeof(uint16_t) + DISCOVERY_PEERS_RESPONSE_CAP * DISCOVERY_WIRE_ENTRY_SIZE];
@@ -226,7 +416,11 @@ void NodeDiscovery_OnGetPeers(node_discovery_t* disc, tcp_connection_t* fromConn
size_t startIdx = total ? (size_t)(random_four_byte() % total) : 0;
for (size_t k = 0; k < total && count < DISCOVERY_PEERS_RESPONSE_CAP; ++k) {
size_t idx = (startIdx + k) % total;
if (haveReq && Discovery_AddrEqual(&all[idx], &reqEndpoint)) continue; // don't tell them about themselves
// Don't tell them about themselves. Matching on identity as well as on the endpoint they
// reached us from matters: a multi-homed peer is known to us under several addresses, and
// handing one of its own back to it is what makes it discover, ping and dial itself.
if (haveReq && Discovery_AddrEqual(&all[idx], &reqEndpoint)) continue;
if (reqNodeId != 0 && allIds[idx] == reqNodeId) continue;
unsigned char entry[DISCOVERY_WIRE_ENTRY_SIZE];
if (!Discovery_AddrToWire(&all[idx], entry)) continue;
memcpy(payload + offset, entry, DISCOVERY_WIRE_ENTRY_SIZE);
@@ -274,6 +468,58 @@ void NodeDiscovery_OnPeersReceived(node_discovery_t* disc, tcp_connection_t* fro
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_RemovePeer(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return;
pthread_mutex_lock(&disc->lock);
size_t n = DynArr_size(disc->peers);
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)DynArr_at(disc->peers, i);
if (Discovery_AddrEqual(&p->addr, endpoint)) {
char ip[INET6_ADDRSTRLEN] = {0};
unsigned short port = 0;
Discovery_AddrToIpPort(&p->addr, ip, sizeof(ip), &port);
printf("NodeDiscovery: struck disconnected peer %s:%u from peer list\n", ip, port);
DynArr_remove(disc->peers, i);
break;
}
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_NoteIdentity(node_discovery_t* disc, const struct sockaddr_storage* endpoint, uint64_t nodeId) {
if (!disc || !endpoint || nodeId == 0) return;
pthread_mutex_lock(&disc->lock);
if (nodeId == localNodeId) {
// The peer on the other end is us under one of our own addresses. Record it and drop it so
// discovery stops treating it as a peer.
Discovery_AddSelfUnlocked(disc, endpoint);
Discovery_RemoveUnlocked(disc, endpoint);
} else {
// Learn the endpoint if we did not already know it - a peer that dialled us is a perfectly
// good discovery candidate, and we now know both its listen endpoint and its identity.
discovered_peer_t* p = Discovery_Upsert(disc, endpoint, 0);
if (p) p->nodeId = nodeId;
}
pthread_mutex_unlock(&disc->lock);
}
void NodeDiscovery_MarkSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return;
pthread_mutex_lock(&disc->lock);
Discovery_AddSelfUnlocked(disc, endpoint);
Discovery_RemoveUnlocked(disc, endpoint);
pthread_mutex_unlock(&disc->lock);
}
int NodeDiscovery_IsSelfEndpoint(node_discovery_t* disc, const struct sockaddr_storage* endpoint) {
if (!disc || !endpoint) return 0;
pthread_mutex_lock(&disc->lock);
int isSelf = Discovery_IsSelfUnlocked(disc, endpoint);
pthread_mutex_unlock(&disc->lock);
return isSelf;
}
// ---- periodic tick -----------------------------------------------------------------------
void NodeDiscovery_Iterate(node_discovery_t* disc) {
@@ -287,10 +533,12 @@ void NodeDiscovery_Iterate(node_discovery_t* disc) {
Node_GetClientList(disc->node, outConns, &outCount);
struct sockaddr_storage outEndpoints[MAX_CONS];
uint64_t outNodeIds[MAX_CONS];
size_t outEpCount = 0;
for (size_t i = 0; i < outCount; ++i) {
struct sockaddr_storage ep;
if (Node_ConnListenEndpoint(outConns[i], &ep)) {
outNodeIds[outEpCount] = Node_ConnPeerNodeId(outConns[i]);
outEndpoints[outEpCount++] = ep;
}
}
@@ -310,6 +558,7 @@ void NodeDiscovery_Iterate(node_discovery_t* disc) {
if (p) {
p->hop = 0;
p->state = DISCOVERY_STATE_CONNECTED;
if (outNodeIds[i] != 0) p->nodeId = outNodeIds[i];
}
}
// Demote entries still marked CONNECTED that are no longer in the outbound set.
@@ -408,17 +657,25 @@ void NodeDiscovery_Iterate(node_discovery_t* disc) {
for (size_t i = 0; i < n; ++i) {
discovered_peer_t* p = (discovered_peer_t*)DynArr_at(disc->peers, i);
if (p->state != DISCOVERY_STATE_REACHABLE) continue;
if (p->lastConnectMs != 0 && (now - p->lastConnectMs) < DISCOVERY_CONNECT_RETRY_MS) continue;
if (!Discovery_ConnectCooledDown(disc, &p->addr, now)) continue;
int already = 0;
for (size_t j = 0; j < outEpCount; ++j) {
if (Discovery_AddrEqual(&p->addr, &outEndpoints[j])) { already = 1; break; }
}
// Skip other addresses of a node we already have an outbound connection to. Only
// outbound counts: an inbound connection from a peer is its own dial, and we still
// want one of our own to it (broadcasts only travel outbound).
if (!already && p->nodeId != 0) {
for (size_t j = 0; j < outEpCount; ++j) {
if (outNodeIds[j] == p->nodeId) { already = 1; break; }
}
}
if (already) continue;
if (!best || p->pingMs < best->pingMs) best = p;
}
if (!best) break;
best->lastConnectMs = now; // reserve so it isn't picked again this tick
Discovery_NoteConnectAttempt(disc, &best->addr, now); // reserve so it isn't picked again this tick
char ip[INET6_ADDRSTRLEN];
unsigned short port = 0;
if (Discovery_AddrToIpPort(&best->addr, ip, sizeof(ip), &port) && port != 0) {
@@ -461,12 +718,28 @@ void NodeDiscovery_PrintPeers(node_discovery_t* disc) {
unsigned short port = 0;
Discovery_AddrToIpPort(&p->addr, ip, sizeof(ip), &port);
const char* stateStr = (p->state <= DISCOVERY_STATE_UNREACHABLE) ? stateNames[p->state] : "?";
if (p->pingMs == UINT64_MAX) {
printf(" %-46s hop=%u state=%-11s ping=--\n", ip, p->hop, stateStr);
char idStr[19];
if (p->nodeId != 0) {
snprintf(idStr, sizeof(idStr), "%016" PRIx64, p->nodeId);
} else {
printf(" %-46s hop=%u state=%-11s ping=%" PRIu64 "ms\n", ip, p->hop, stateStr, p->pingMs);
snprintf(idStr, sizeof(idStr), "%-16s", "?");
}
if (p->pingMs == UINT64_MAX) {
printf(" %-46s hop=%u state=%-11s id=%s ping=--\n", ip, p->hop, stateStr, idStr);
} else {
printf(" %-46s hop=%u state=%-11s id=%s ping=%" PRIu64 "ms\n", ip, p->hop, stateStr, idStr, p->pingMs);
}
(void)port; // port is part of ip endpoint identity; shown via connect logs
}
size_t selfCount = DynArr_size(disc->selfEndpoints);
printf("Own endpoints (%zu):\n", selfCount);
for (size_t i = 0; i < selfCount; ++i) {
const struct sockaddr_storage* self = (const struct sockaddr_storage*)DynArr_at(disc->selfEndpoints, i);
char ip[INET6_ADDRSTRLEN] = {0};
unsigned short port = 0;
Discovery_AddrToIpPort(self, ip, sizeof(ip), &port);
printf(" %-46s port=%u\n", ip, port);
}
pthread_mutex_unlock(&disc->lock);
}
src/nets/orphan_pool.c
+529 -151
View File
@@ -1,5 +1,7 @@
#include <nets/orphan_pool.h>
#include <constants.h>
#include <dynarr.h>
#include <pthread.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
@@ -7,17 +9,38 @@
typedef struct {
block_t* block;
uint64_t height;
uint64_t observedAtTipHeight; // local tip height when first seen; stamped once (reorg penalty)
uint64_t sequence; // insertion order, used to evict the oldest entry when full
uint8_t hash[32];
} orphan_entry_t;
static DynArr* g_orphans = NULL;
static uint64_t g_nextSequence = 0;
// The pool is touched by the maintenance thread, by every per-peer TCP thread and by the REPL
// thread. It used to have no synchronisation at all, so a concurrent Insert could realloc the
// array out from under a scan that was holding a raw element pointer.
//
// Lock ordering: this mutex is never held while calling into chain.c (which takes chainLock).
// Candidate branches are collected under the lock, the lock is dropped, and only then is
// Chain_ReplaceBranch/Chain_AddBlock called.
static pthread_mutex_t g_orphanLock = PTHREAD_MUTEX_INITIALIZER;
static void OrphanPool_InitLocked(void) {
if (!g_orphans) {
g_orphans = DYNARR_CREATE(orphan_entry_t, 16);
}
}
void OrphanPool_Init(void) {
if (g_orphans) return;
g_orphans = DYNARR_CREATE(orphan_entry_t, 16);
pthread_mutex_lock(&g_orphanLock);
OrphanPool_InitLocked();
pthread_mutex_unlock(&g_orphanLock);
}
void OrphanPool_Destroy(void) {
if (!g_orphans) return;
pthread_mutex_lock(&g_orphanLock);
if (g_orphans) {
size_t n = DynArr_size(g_orphans);
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
@@ -28,181 +51,536 @@ void OrphanPool_Destroy(void) {
DynArr_destroy(g_orphans);
g_orphans = NULL;
}
void OrphanPool_Insert(block_t* block, uint64_t height) {
if (!block) return;
if (!g_orphans) OrphanPool_Init();
orphan_entry_t e;
e.block = block;
e.height = height;
(void)DynArr_push_back(g_orphans, &e);
pthread_mutex_unlock(&g_orphanLock);
}
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain, uint64_t forkHeight) {
if (!g_orphans || !chain) return 0;
DynArr* seq = DYNARR_CREATE(block_t*, 8);
if (!seq) return 0;
size_t cursor = forkHeight;
while (1) {
bool found = false;
size_t count = DynArr_size(g_orphans);
for (size_t i = 0; i < count; ++i) {
orphan_entry_t* entry = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (!entry || !entry->block) continue;
if (entry->height == cursor) {
(void)DynArr_push_back(seq, &entry->block);
found = true;
break;
}
}
if (!found) break;
cursor++;
static ssize_t OrphanPool_FindByHashLocked(const uint8_t blockHash[32]) {
if (!g_orphans || !blockHash) {
return -1;
}
size_t seqCount = DynArr_size(seq);
if (seqCount == 0) {
DynArr_destroy(seq);
return 0;
}
size_t currentTipHeight = Chain_Size(chain) == 0 ? 0 : Chain_Size(chain) - 1;
size_t seqTopHeight = forkHeight + seqCount - 1;
if (seqTopHeight <= currentTipHeight) {
DynArr_destroy(seq);
return 0;
}
size_t rollbackHeight = (forkHeight == 0) ? 0 : (forkHeight - 1);
if (!Chain_RollbackToHeight(chain, rollbackHeight)) {
DynArr_destroy(seq);
return 0;
}
size_t attached = 0;
for (size_t i = 0; i < seqCount; ++i) {
block_t* bptr = *(block_t**)DynArr_at(seq, i);
if (!bptr || !Chain_AddBlock(chain, bptr)) {
break;
}
size_t count = DynArr_size(g_orphans);
for (size_t j = 0; j < count; ++j) {
orphan_entry_t* entry = (orphan_entry_t*)DynArr_at(g_orphans, j);
if (entry && entry->block == bptr) {
DynArr_remove(g_orphans, j);
break;
}
}
attached++;
}
DynArr_destroy(seq);
return attached;
}
size_t OrphanPool_AttemptAttach(blockchain_t* chain) {
if (!g_orphans || !chain) return 0;
size_t attached = 0;
bool madeProgress = true;
// Attempt repeatedly while progress is made (to handle chained orphans)
while (madeProgress) {
madeProgress = false;
size_t n = DynArr_size(g_orphans);
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (!e || !e->block) continue;
uint64_t parentIndex = (e->height == 0) ? (uint64_t)-1 : (e->height - 1);
bool parentExists = false;
if (e->height == 0) {
// genesis-style block: parent is zero-hash; accept if chain empty
parentExists = (Chain_Size(chain) == 0);
} else if (parentIndex < Chain_Size(chain)) {
block_t* parent = NULL;
if (Chain_GetBlockCopy(chain, (size_t)parentIndex, &parent) && parent) {
parentExists = true;
Block_Destroy(parent);
} else {
parentExists = false;
if (e && memcmp(e->hash, blockHash, 32) == 0) {
return (ssize_t)i;
}
}
if (parentExists) {
if (e->height < Chain_Size(chain)) {
block_t* local = NULL;
if (Chain_GetBlockCopy(chain, (size_t)e->height, &local) && local) {
uint8_t localHash[32];
return -1;
}
// Drop the entry with the lowest sequence number, so a flood of unusable orphans cannot grow
// without bound. Returns true if something was evicted.
static bool OrphanPool_EvictOldestLocked(void) {
if (!g_orphans) {
return false;
}
size_t n = DynArr_size(g_orphans);
if (n == 0) {
return false;
}
size_t oldestIndex = 0;
uint64_t oldestSequence = UINT64_MAX;
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (e && e->sequence < oldestSequence) {
oldestSequence = e->sequence;
oldestIndex = i;
}
}
orphan_entry_t* victim = (orphan_entry_t*)DynArr_at(g_orphans, oldestIndex);
if (victim && victim->block) {
Block_Destroy(victim->block);
}
DynArr_remove(g_orphans, oldestIndex);
return true;
}
void OrphanPool_Insert(block_t* block, uint64_t height, uint64_t observedAtTipHeight) {
if (!block) {
return;
}
uint8_t blockHash[32];
Block_CalculateHash(block, blockHash);
pthread_mutex_lock(&g_orphanLock);
OrphanPool_InitLocked();
if (!g_orphans) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block);
return;
}
// Reject duplicates. The same block reaches us from every peer that relays it, and without
// this each copy became its own permanently-resident entry.
if (OrphanPool_FindByHashLocked(blockHash) >= 0) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block);
return;
}
while (DynArr_size(g_orphans) >= MAX_ORPHAN_BLOCKS) {
if (!OrphanPool_EvictOldestLocked()) {
break;
}
}
orphan_entry_t e;
memset(&e, 0, sizeof(e));
e.block = block;
e.height = height;
e.observedAtTipHeight = observedAtTipHeight;
e.sequence = g_nextSequence++;
memcpy(e.hash, blockHash, 32);
if (!DynArr_push_back(g_orphans, &e)) {
pthread_mutex_unlock(&g_orphanLock);
Block_Destroy(block);
return;
}
pthread_mutex_unlock(&g_orphanLock);
}
bool OrphanPool_Contains(const uint8_t blockHash[32]) {
pthread_mutex_lock(&g_orphanLock);
bool found = OrphanPool_FindByHashLocked(blockHash) >= 0;
pthread_mutex_unlock(&g_orphanLock);
return found;
}
size_t OrphanPool_Size(void) {
pthread_mutex_lock(&g_orphanLock);
size_t n = g_orphans ? DynArr_size(g_orphans) : 0;
pthread_mutex_unlock(&g_orphanLock);
return n;
}
// Remove the entry with this hash without freeing the block, and hand the block back. Used once a
// block has been given to the chain, which then owns its transaction array.
static block_t* OrphanPool_TakeByHashLocked(const uint8_t blockHash[32]) {
ssize_t index = OrphanPool_FindByHashLocked(blockHash);
if (index < 0) {
return NULL;
}
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, (size_t)index);
block_t* blk = e ? e->block : NULL;
DynArr_remove(g_orphans, (size_t)index);
return blk;
}
static void OrphanPool_DropByHashLocked(const uint8_t blockHash[32]) {
ssize_t index = OrphanPool_FindByHashLocked(blockHash);
if (index < 0) {
return;
}
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, (size_t)index);
if (e && e->block) {
Block_Destroy(e->block);
}
DynArr_remove(g_orphans, (size_t)index);
}
/**
* Copy out the orphan that extends `prevHash` at `height`, if any.
* Returns false when there is no such orphan. Caller must hold the pool lock.
**/
static bool OrphanPool_FindChildLocked(uint64_t height,
const uint8_t prevHash[32],
block_t** outBlock,
uint64_t* outObservedAtTipHeight,
uint8_t outHash[32]) {
if (!g_orphans) {
return false;
}
size_t n = DynArr_size(g_orphans);
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (!e || !e->block) {
continue;
}
if (e->height != height) {
continue;
}
if (memcmp(e->block->header.prevHash, prevHash, 32) != 0) {
continue;
}
*outBlock = e->block;
*outObservedAtTipHeight = e->observedAtTipHeight;
memcpy(outHash, e->hash, 32);
return true;
}
return false;
}
/**
* Follow prevHash links from `forkHeight` to build the longest branch the pool can offer.
*
* The old implementation took the first orphan found at each successive height with no linkage
* check at all, which could splice blocks from two different forks into one incoherent branch.
* Caller must hold the pool lock. The returned array borrows the pooled block pointers; the pool
* still owns them (Chain_ReplaceBranch applies copies).
**/
static size_t OrphanPool_CollectBranchLocked(uint64_t forkHeight,
const uint8_t forkParentHash[32],
block_t*** outBlocks,
uint8_t** outHashes,
uint64_t* outObservedAtTipHeight) {
*outBlocks = NULL;
*outHashes = NULL;
*outObservedAtTipHeight = 0;
DynArr* collected = DYNARR_CREATE(block_t*, 8);
DynArr* hashes = DYNARR_CREATE(uint8_t, 8 * 32);
if (!collected || !hashes) {
if (collected) DynArr_destroy(collected);
if (hashes) DynArr_destroy(hashes);
return 0;
}
uint8_t expectedPrevHash[32];
memcpy(expectedPrevHash, forkParentHash, 32);
uint64_t earliestObserved = UINT64_MAX;
uint64_t cursor = forkHeight;
size_t count = 0;
while (1) {
block_t* child = NULL;
uint64_t observed = 0;
uint8_t childHash[32];
if (!OrphanPool_FindChildLocked(cursor, expectedPrevHash, &child, &observed, childHash)) {
break;
}
if (!DynArr_push_back(collected, &child)) {
break;
}
for (size_t b = 0; b < 32; ++b) {
if (!DynArr_push_back(hashes, &childHash[b])) {
break;
}
}
if (observed < earliestObserved) {
earliestObserved = observed;
}
memcpy(expectedPrevHash, childHash, 32);
cursor++;
count++;
}
if (count == 0) {
DynArr_destroy(collected);
DynArr_destroy(hashes);
return 0;
}
block_t** blocks = (block_t**)calloc(count, sizeof(block_t*));
uint8_t* hashOut = (uint8_t*)calloc(count, 32);
if (!blocks || !hashOut) {
free(blocks);
free(hashOut);
DynArr_destroy(collected);
DynArr_destroy(hashes);
return 0;
}
for (size_t i = 0; i < count; ++i) {
blocks[i] = *(block_t**)DynArr_at(collected, i);
for (size_t b = 0; b < 32; ++b) {
hashOut[i * 32 + b] = *(uint8_t*)DynArr_at(hashes, i * 32 + b);
}
}
DynArr_destroy(collected);
DynArr_destroy(hashes);
*outBlocks = blocks;
*outHashes = hashOut;
*outObservedAtTipHeight = earliestObserved == UINT64_MAX ? 0ULL : earliestObserved;
return count;
}
// Discard orphans that can no longer ever be applied: anything at or below the current tip whose
// hash does not match the block we actually have there. Without this the pool only ever grew, and
// permanently-invalid entries were retried on every 1 Hz maintenance tick.
static void OrphanPool_PruneStale(blockchain_t* chain) {
if (!chain) {
return;
}
const size_t chainSize = Chain_Size(chain);
// Collect the hashes to drop first, so we never call into chain.c while holding the pool lock.
DynArr* doomed = DYNARR_CREATE(uint8_t, 32);
if (!doomed) {
return;
}
pthread_mutex_lock(&g_orphanLock);
size_t n = g_orphans ? DynArr_size(g_orphans) : 0;
DynArr* candidates = DYNARR_CREATE(uint8_t, 32);
DynArr* candidateHeights = DYNARR_CREATE(uint64_t, 8);
if (candidates && candidateHeights) {
for (size_t i = 0; i < n; ++i) {
orphan_entry_t* e = (orphan_entry_t*)DynArr_at(g_orphans, i);
if (!e || !e->block) {
continue;
}
if (e->height >= (uint64_t)chainSize) {
continue; // still ahead of us; may attach later
}
for (size_t b = 0; b < 32; ++b) {
(void)DynArr_push_back(candidates, &e->hash[b]);
}
(void)DynArr_push_back(candidateHeights, &e->height);
}
}
pthread_mutex_unlock(&g_orphanLock);
size_t candidateCount = candidateHeights ? DynArr_size(candidateHeights) : 0;
for (size_t i = 0; i < candidateCount; ++i) {
uint64_t height = *(uint64_t*)DynArr_at(candidateHeights, i);
uint8_t orphanHash[32];
for (size_t b = 0; b < 32; ++b) {
orphanHash[b] = *(uint8_t*)DynArr_at(candidates, i * 32 + b);
}
block_t* local = NULL;
if (!Chain_GetBlockCopy(chain, (size_t)height, &local) || !local) {
continue;
}
uint8_t localHash[32];
Block_CalculateHash(local, localHash);
Block_CalculateHash(e->block, orphanHash);
Block_Destroy(local);
if (memcmp(localHash, orphanHash, 32) != 0) {
size_t adopted = OrphanPool_TryAdoptBranch(chain, e->height);
if (adopted > 0) {
attached += adopted;
madeProgress = true;
n = DynArr_size(g_orphans);
i = (size_t)-1;
// Same block we already have: pure duplicate, drop it. A different block at a height we
// have already passed is kept, because it may yet be the base of a heavier branch.
if (memcmp(localHash, orphanHash, 32) == 0) {
for (size_t b = 0; b < 32; ++b) {
(void)DynArr_push_back(doomed, &orphanHash[b]);
}
}
}
size_t doomedCount = DynArr_size(doomed) / 32;
if (doomedCount > 0) {
pthread_mutex_lock(&g_orphanLock);
for (size_t i = 0; i < doomedCount; ++i) {
uint8_t h[32];
for (size_t b = 0; b < 32; ++b) {
h[b] = *(uint8_t*)DynArr_at(doomed, i * 32 + b);
}
OrphanPool_DropByHashLocked(h);
}
pthread_mutex_unlock(&g_orphanLock);
}
if (candidates) DynArr_destroy(candidates);
if (candidateHeights) DynArr_destroy(candidateHeights);
DynArr_destroy(doomed);
}
/**
* Try to extend the current tip directly with pooled orphans.
* Returns the number of blocks attached.
**/
static size_t OrphanPool_ExtendTip(blockchain_t* chain) {
size_t attached = 0;
while (1) {
const size_t chainSize = Chain_Size(chain);
uint8_t tipHash[32];
memset(tipHash, 0, sizeof(tipHash));
if (chainSize > 0) {
block_t* tip = NULL;
if (!Chain_GetBlockCopy(chain, chainSize - 1, &tip) || !tip) {
break;
}
}
} else if (local) {
Block_Destroy(local);
}
Block_CalculateHash(tip, tipHash);
Block_Destroy(tip);
}
// Verify that the parent's hash matches the orphan's prevHash before attaching.
bool parentMatches = false;
if (e->height == 0) {
parentMatches = (Chain_Size(chain) == 0);
} else {
block_t* parent = NULL;
if (Chain_GetBlockCopy(chain, (size_t)parentIndex, &parent) && parent) {
uint8_t parentHash[32];
Block_CalculateHash(parent, parentHash);
parentMatches = (memcmp(parentHash, e->block->header.prevHash, 32) == 0);
Block_Destroy(parent);
} else {
parentMatches = false;
}
}
// Take a copy of the candidate under the lock, then release it before touching the chain.
pthread_mutex_lock(&g_orphanLock);
block_t* pooled = NULL;
uint64_t observed = 0;
uint8_t candidateHash[32];
bool found = OrphanPool_FindChildLocked((uint64_t)chainSize, tipHash, &pooled, &observed, candidateHash);
block_t* candidate = found ? Block_Copy(pooled) : NULL;
pthread_mutex_unlock(&g_orphanLock);
if (!parentMatches) {
// Parent exists but does not match this orphan's prevHash.
size_t adopted = OrphanPool_TryAdoptBranch(chain, e->height);
if (adopted > 0) {
attached += adopted;
madeProgress = true;
n = DynArr_size(g_orphans);
i = (size_t)-1;
if (!found) {
break;
}
if (!candidate) {
break;
}
if (!Chain_AddBlock(chain, candidate)) {
// Permanent rejection for this block at this height (bad coinbase, wrong difficulty,
// ...). Drop it rather than retrying it on every maintenance tick forever.
Block_Destroy(candidate);
pthread_mutex_lock(&g_orphanLock);
OrphanPool_DropByHashLocked(candidateHash);
pthread_mutex_unlock(&g_orphanLock);
continue;
}
// Try to add to chain
if (Chain_AddBlock(chain, e->block)) {
// Chain_AddBlock took ownership of the transaction array and cleared our pointer to it.
Block_Destroy(candidate);
pthread_mutex_lock(&g_orphanLock);
block_t* taken = OrphanPool_TakeByHashLocked(candidateHash);
pthread_mutex_unlock(&g_orphanLock);
if (taken) {
Block_Destroy(taken); // the pool's own copy is independent of the one we applied
}
attached++;
madeProgress = true;
// remove this entry
DynArr_remove(g_orphans, i);
// adjust indices
n = DynArr_size(g_orphans);
i = (size_t)-1; // reset outer loop
break;
} else {
// Keep the orphan around; rejection may be temporary while the local tip is being reorged.
continue;
}
}
}
}
return attached;
}
/**
* Look for a competing branch that forks below our tip and is worth adopting.
* The work comparison, the reorg penalty and the atomicity all live in Chain_ReplaceBranch.
**/
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain, bool bypassPenalty) {
const size_t chainSize = Chain_Size(chain);
if (chainSize == 0) {
return 0;
}
// Walk fork points from just below the tip downwards; the shallowest fork wins, which is also
// the one with the smallest reorg penalty.
for (size_t forkHeight = chainSize; forkHeight >= 1; --forkHeight) {
block_t* parent = NULL;
if (!Chain_GetBlockCopy(chain, forkHeight - 1, &parent) || !parent) {
continue;
}
uint8_t parentHash[32];
Block_CalculateHash(parent, parentHash);
Block_Destroy(parent);
pthread_mutex_lock(&g_orphanLock);
block_t** branch = NULL;
uint8_t* branchHashes = NULL;
uint64_t observedAtTipHeight = 0;
size_t branchCount = OrphanPool_CollectBranchLocked((uint64_t)forkHeight, parentHash,
&branch, &branchHashes, &observedAtTipHeight);
// Copy the branch so the pool lock can be released before we call into the chain.
block_t** branchCopies = NULL;
if (branchCount > 0) {
branchCopies = (block_t**)calloc(branchCount, sizeof(block_t*));
if (branchCopies) {
for (size_t i = 0; i < branchCount; ++i) {
branchCopies[i] = Block_Copy(branch[i]);
}
}
}
pthread_mutex_unlock(&g_orphanLock);
free(branch);
if (branchCount == 0 || !branchCopies) {
free(branchHashes);
if (branchCopies) {
free(branchCopies);
}
if (forkHeight == 1) break;
continue;
}
bool copiedAll = true;
for (size_t i = 0; i < branchCount; ++i) {
if (!branchCopies[i]) {
copiedAll = false;
}
}
bool adopted = false;
if (copiedAll) {
adopted = Chain_ReplaceBranch(chain, forkHeight, branchCopies, branchCount, observedAtTipHeight,
bypassPenalty);
}
for (size_t i = 0; i < branchCount; ++i) {
if (branchCopies[i]) {
Block_Destroy(branchCopies[i]); // Chain_ReplaceBranch applied its own copies
}
}
free(branchCopies);
if (adopted) {
printf("Adopted competing branch of %zu block(s) at fork height %zu\n", branchCount, forkHeight);
pthread_mutex_lock(&g_orphanLock);
for (size_t i = 0; i < branchCount; ++i) {
OrphanPool_DropByHashLocked(&branchHashes[i * 32]);
}
pthread_mutex_unlock(&g_orphanLock);
free(branchHashes);
return branchCount;
}
free(branchHashes);
if (forkHeight == 1) {
break;
}
}
return 0;
}
size_t OrphanPool_AttemptAttach(blockchain_t* chain) {
return OrphanPool_AttemptAttachForced(chain, false);
}
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty) {
if (!chain) {
return 0;
}
pthread_mutex_lock(&g_orphanLock);
bool empty = (g_orphans == NULL) || (DynArr_size(g_orphans) == 0);
pthread_mutex_unlock(&g_orphanLock);
if (empty) {
return 0;
}
size_t attached = 0;
// Extending the tip is always preferable to a reorg, so try that to exhaustion first, and only
// then consider replacing part of our chain with a competing branch.
while (1) {
size_t extended = OrphanPool_ExtendTip(chain);
attached += extended;
size_t adopted = OrphanPool_TryAdoptBranch(chain, bypassPenalty);
attached += adopted;
if (extended == 0 && adopted == 0) {
break;
}
}
OrphanPool_PruneStale(chain);
return attached;
}
src/numgen.c
+31
View File
@@ -1,5 +1,8 @@
#include <numgen.h>
#include <stdio.h>
#include <unistd.h>
unsigned char random_byte(void) {
return (unsigned char)(rand() % 256);
}
@@ -39,3 +42,31 @@ uint64_t random_eight_byte(void) {
return x;
}
uint64_t random_secure_eight_byte(void) {
uint64_t x = 0;
FILE* urandom = fopen("/dev/urandom", "rb");
if (urandom) {
size_t got = fread(&x, 1, sizeof(x), urandom);
fclose(urandom);
if (got == sizeof(x) && x != 0) {
return x;
}
}
// Fallback: srand() is seeded from the wall clock in whole seconds, so two nodes launched
// together would draw identical values. Mix in the pid and the sub-second clock to separate them.
struct timespec ts;
if (clock_gettime(CLOCK_REALTIME, &ts) != 0) {
ts.tv_sec = 0;
ts.tv_nsec = 0;
}
x = random_eight_byte();
x ^= (uint64_t)ts.tv_nsec;
x ^= ((uint64_t)ts.tv_sec) << 16;
x ^= ((uint64_t)getpid()) << 40;
return x ? x : 1; // 0 means "no identity advertised" on the wire
}
src/tcpd/tcpconnection.c
+15
View File
@@ -31,6 +31,7 @@ int TcpConnection_Init(tcp_connection_t* conn, int sockFd, const struct sockaddr
conn->closing = false;
conn->disconnectedNotified = false;
atomic_init(&conn->pinCount, 0);
conn->dataBuf = NULL;
conn->dataBufLen = 0;
conn->dataBufCap = 0;
@@ -262,6 +263,20 @@ bool TcpConnection_IsDisconnectNotified(tcp_connection_t* conn) {
return notified;
}
void TcpConnection_Pin(tcp_connection_t* conn) {
if (!conn) {
return;
}
atomic_fetch_add(&conn->pinCount, 1);
}
void TcpConnection_Unpin(tcp_connection_t* conn) {
if (!conn) {
return;
}
atomic_fetch_sub(&conn->pinCount, 1);
}
static int extract_v4(const tcp_connection_t* conn, struct in_addr* v4out) {
if (conn->addrFamily == AF_INET6) {
const struct sockaddr_in6* a6 = (const struct sockaddr_in6*)&conn->peerAddr;
src/tcpd/tcpserver.c
+81 -15
View File
@@ -16,15 +16,20 @@ typedef struct {
int listenFd;
} tcpaccept_thread_args_t;
static void TcpServer_RemoveClientByPtrUnlocked(tcp_server_t* svr, tcp_connection_t* cli) {
// Returns non-zero if `cli` was still registered (and has now been unregistered). A zero return
// means someone else already claimed the slot -- see the detach logic in the client thread.
static int TcpServer_RemoveClientByPtrUnlocked(tcp_server_t* svr, tcp_connection_t* cli) {
if (!svr || !svr->clientsArrPtr || !cli) {
return;
return 0;
}
size_t idx = Generic_FindClientInArrayByPtr(svr->clientsArrPtr, cli, svr->maxClients);
if (idx != SIZE_MAX) {
svr->clientsArrPtr[idx] = NULL;
return 1;
}
return 0;
}
static void* TcpServer_clientthreadprocess(void* ptr) {
@@ -65,16 +70,38 @@ static void* TcpServer_clientthreadprocess(void* ptr) {
cli->on_disconnect(cli);
}
// Unregister, decide who joins us, and free -- all under clientsMutex.
//
// The destroy/free used to happen after the lock was released, which left a window where
// TcpServer_Stop could be holding this very pointer and about to use it. Doing it under the
// same lock Stop uses to inspect the slots removes that window entirely.
pthread_mutex_lock(&svr->clientsMutex);
TcpServer_RemoveClientByPtrUnlocked(svr, cli);
pthread_mutex_unlock(&svr->clientsMutex);
// If our slot was still ours, TcpServer_Stop has not claimed us and never will (we are leaving
// the array now), so nobody is going to join this thread -- detach it or its resources leak.
// If the slot was already cleared, Stop took our handle and is waiting in pthread_join, so we
// must stay joinable.
if (TcpServer_RemoveClientByPtrUnlocked(svr, cli)) {
pthread_detach(pthread_self());
}
TcpConnection_Destroy(cli);
free(cli);
pthread_mutex_unlock(&svr->clientsMutex);
return NULL;
}
// listenFd is borrowed, not owned: it is ptr->sockFd / ptr->sockFdV4, handed
// over by TcpServer_Start and closed by TcpServer_Stop. GCC's -fanalyzer infers
// from accept() that the fd is open and then holds this function responsible
// for closing it, so it reports a leak on every path that leaves the loop.
// Clang does not know this warning group, hence the guard.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
static void* TcpServer_threadprocess(void* ptr) {
tcpaccept_thread_args_t* args = (tcpaccept_thread_args_t*)ptr;
if (!args || !args->serverPtr) {
@@ -170,6 +197,9 @@ static void* TcpServer_threadprocess(void* ptr) {
return NULL;
}
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
tcp_server_t* TcpServer_Create() {
tcp_server_t* svr = (tcp_server_t*)malloc(sizeof(*svr));
@@ -208,6 +238,16 @@ void TcpServer_Destroy(tcp_server_t* ptr) {
free(ptr);
}
// Both sockets are handed to the caller through ptr->sockFd / ptr->sockFdV4 and
// closed by TcpServer_Stop, so neither leaks. -fanalyzer loses track of the
// first store across the second socket's branches and reports it anyway; the
// report is positional, not semantic — swapping the IPv6 and IPv4 blocks moves
// the warning from fd6 to fd4, and deleting the unrelated second block silences
// it entirely.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
void TcpServer_Init(tcp_server_t* ptr, unsigned short port, const char* addr) {
if (!ptr || !addr) {
return;
@@ -255,7 +295,18 @@ void TcpServer_Init(tcp_server_t* ptr, unsigned short port, const char* addr) {
}
}
}
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
// Same borrowed-fd false positive as TcpServer_threadprocess: listen() teaches
// -fanalyzer that ptr->sockFd / ptr->sockFdV4 are open passive sockets, so it
// expects this function to close them. They belong to the tcp_server_t and are
// closed by TcpServer_Stop.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
void TcpServer_Start(tcp_server_t* ptr, int maxcons) {
if (!ptr || (ptr->sockFd < 0 && ptr->sockFdV4 < 0) || maxcons <= 0 || ptr->isRunning) {
return;
@@ -329,6 +380,9 @@ void TcpServer_Start(tcp_server_t* ptr, int maxcons) {
pthread_mutex_unlock(&ptr->clientsMutex);
}
}
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
void TcpServer_Stop(tcp_server_t* ptr) {
if (!ptr || !ptr->isRunning) {
@@ -359,30 +413,42 @@ void TcpServer_Stop(tcp_server_t* ptr) {
}
ptr->svrThreadV4 = 0;
// Ask every live client to close and copy out its thread handle, all under clientsMutex.
//
// This used to read the client slots with the lock released, which races with an exiting client
// thread clearing its own slot -- and worse, that thread destroys and frees the connection right
// afterwards, so the pointer read here could already be freed memory. Copying the pthread_t
// while holding the lock means the join below never dereferences the connection at all, and the
// client thread cannot free itself out from under us because it does that under the same lock.
pthread_mutex_lock(&ptr->clientsMutex);
size_t maxClients = ptr->maxClients;
tcp_connection_t** local = ptr->clientsArrPtr;
pthread_mutex_unlock(&ptr->clientsMutex);
pthread_t* joinHandles = maxClients ? (pthread_t*)calloc(maxClients, sizeof(pthread_t)) : NULL;
size_t joinCount = 0;
if (ptr->clientsArrPtr) {
for (size_t i = 0; i < maxClients; ++i) {
tcp_connection_t* cli = local[i];
tcp_connection_t* cli = ptr->clientsArrPtr[i];
if (!cli) {
continue;
}
TcpConnection_RequestClose(cli);
}
for (size_t i = 0; i < maxClients; ++i) {
tcp_connection_t* cli = local[i];
if (!cli) {
continue;
if (joinHandles && !pthread_equal(cli->ioThread, pthread_self())) {
joinHandles[joinCount++] = cli->ioThread;
// Claim the slot: the client thread checks whether it is still registered to decide
// whether to detach itself or stay joinable for the pthread_join below.
ptr->clientsArrPtr[i] = NULL;
}
}
}
pthread_mutex_unlock(&ptr->clientsMutex);
if (!pthread_equal(cli->ioThread, pthread_self())) {
pthread_join(cli->ioThread, NULL);
}
// Join outside the lock: a client thread needs clientsMutex to finish unregistering itself.
for (size_t i = 0; i < joinCount; ++i) {
pthread_join(joinHandles[i], NULL);
}
free(joinHandles);
pthread_mutex_lock(&ptr->clientsMutex);
free(ptr->clientsArrPtr);
src/txmempool.c
+47
View File
@@ -1,4 +1,5 @@
#include <txmempool.h>
#include <constants.h>
#include <pthread.h>
static pthread_mutex_t g_txMempoolLock;
@@ -12,6 +13,52 @@ void TxMempool_Init() {
g_txMempoolLockInitialized = true;
}
bool TxMempool_PolicyAccepts(const signed_transaction_t* tx, uint64_t nowMs) {
if (!tx) {
return false;
}
const uint64_t ts = tx->transaction.timestamp;
// Dated too far in the future, measured against OUR CLOCK rather than the chain tip -- see the
// note in the header. Refusing this also limits the one real footgun in the replay guard: a
// wildly future timestamp permanently advances that account's lastTxTimestamp and locks it out
// until real time catches up.
if (ts > nowMs && (ts - nowMs) > TX_MAX_FUTURE_DRIFT_MS) {
return false;
}
// Too old to be worth holding. Not a validity judgement -- just pool hygiene.
if (nowMs > ts && (nowMs - ts) > TX_EXPIRY_MS) {
return false;
}
return true;
}
size_t TxMempool_PruneExpired(uint64_t nowMs) {
if (!txMempool) {
return 0;
}
size_t removed = 0;
pthread_mutex_lock(&g_txMempoolLock);
for (khiter_t k = kh_begin(txMempool); k != kh_end(txMempool); ++k) {
if (!kh_exist(txMempool, k)) {
continue;
}
const uint64_t ts = kh_value(txMempool, k).transaction.timestamp;
if (nowMs > ts && (nowMs - ts) > TX_EXPIRY_MS) {
kh_del(tx_mempool_map_m, txMempool, k);
removed++;
}
}
pthread_mutex_unlock(&g_txMempoolLock);
return removed;
}
int TxMempool_Insert(signed_transaction_t tx) {
if (!txMempool) { return -1; }
src/udpd/udpnode.c
+11
View File
@@ -181,6 +181,14 @@ static void* UdpNode_RetryThreadProc(void* arg) {
return NULL;
}
// Same borrowed/escaped-fd false positive as TcpServer_Init: both sockets are
// handed to the caller through node->sockFd / node->sockFdV4 and closed by
// UdpNode_Stop. -fanalyzer loses the first store across the second socket's
// branches and reports it as a leak.
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wanalyzer-fd-leak"
#endif
int UdpNode_Init(udp_node_t* node, uint16_t port) {
if (!node) {
return -1;
@@ -237,6 +245,9 @@ int UdpNode_Init(udp_node_t* node, uint16_t port) {
pthread_mutex_init(&node->pingsMutex, NULL);
return 0;
}
#if defined(__GNUC__) && !defined(__clang__)
#pragma GCC diagnostic pop
#endif
void UdpNode_SetCallbacks(udp_node_t* node,
void (*on_pong)(udp_node_t*, const struct sockaddr_storage*, uint64_t, int, uint64_t, void*),