Let a node that is behind adopt the peer's branch, and fix two bugs that only appear once a reorg actually applies

Reported: a node at height N+10 on its own short fork, with the peer at N+500,
could never sync. It located the fork point correctly, pooled the branch, and
then refused it forever. At 90s blocks a node is 960 blocks behind after a day
offline, so this is the normal case, not an edge case.

== Why it could never recover ==

The reorg delay was served by LOCAL chain growth alone:

    elapsed = tipHeight - observedTip

That only makes sense for a node whose tip is advancing. A node that is behind
has a frozen tip precisely because it is rejecting the branch, so elapsed stays
0 forever while penalty(5) = 42. Both escape hatches also fail: mining out of it
means extending a fork nobody accepts, and the IBD exemption never fires for a
node that is mining, because mining keeps its tip fresh.

Being 490 blocks behind is being behind, not a reorg contest. Refusing to adopt
protects nothing while the node falls a further 40 blocks behind every hour.

The delay is now satisfiable by EITHER side making progress:

    localGrowth = tipHeight - observedTip
    branchLead  = candidateTip - tipHeight        (0 if not ahead)
    elapsed     = max(localGrowth, branchLead)

A branch already extending `penalty` blocks past our tip has demonstrated
exactly what the delay asks for, and every one of those blocks carries PoW we
validated ourselves. Requiring us to independently produce the same amount is
demanding the same proof twice. Only VALIDATED blocks count -- a peer's
advertised height is not evidence and never reaches this code.

It degrades correctly in both directions: a mining node's tip advances, so an
attacker must outpace it by penalty blocks; a node that is only observing
follows the heaviest chain, which is what an observer should do.

== The window has to keep pulling the branch ==

Every forked delivery reset nextReq back to our own tip, so the window
re-requested the same eight heights forever and the pool never grew past the
window size -- branchLead could not rise even in principle. The backward walk
now runs ONCE to establish linkage, then the window marches forward pooling the
branch, retrying adoption per window-full with a final attempt when it drains.

== sync force ==

Operator override that skips the delay for one sync, for a node whose chain is
known to be the wrong one. Threaded explicitly (Chain_ReplaceBranch gains
bypassPenalty, OrphanPool_AttemptAttachForced) rather than through a global, so
nothing a peer sends can reach it. Linkage, work comparison and atomicity still
apply -- it waives only the waiting, and says so loudly in the log.

== Bug found in testing: PoW is branch-relative ==

A block's epoch seed is the last block of the previous epoch ON ITS OWN BRANCH.
Validating a competing branch's block against OUR epoch seed does not merely
fail to resolve: when the chains diverge before the boundary it resolves to the
WRONG seed and rejects a perfectly valid block. Any fork spanning an epoch
boundary was therefore impossible to assemble -- the branch could not grow past
the boundary block, so branchLead stalled one short of it.

The receive path now does self-contained checks only (Block_HasValidStructure:
merkle, transactions, vote, non-empty). Proof of work moved to
Chain_AddBlockLocked, at the point a block joins the chain, where the branch
context is real -- the rollback has put its ancestors in place by then. That is
where the invariant belongs and it removes a duplicate check rather than adding
one. Needs Chain_DagParamsForHeightLocked, because Chain_AddBlockLocked already
holds chainLock for writing and the lock is not recursive.

Consequence worth knowing: the orphan pool can now hold blocks whose work has
not been verified, bounded by MAX_ORPHAN_BLOCKS (512). Each still had to pass
merkle and full transaction/signature validation, and none can reach the chain
unverified.

This bug also affected plain forward sync across block 350000; it was masked
because appending keeps the boundary block in the chain.

== Bug found in testing: stale DAG accepted as current ==

Block_PowHashHeavy matched on epoch index and size but not the seed. A DAG's
content is a function of (seed, size); the epoch index is a label for it. A
reorg is exactly the event that changes the seed while leaving index and size
untouched, so mid-apply the miner's context still held a DAG built from the
PRE-reorg seed, the guard passed, and a valid block was hashed against the wrong
lanes. g_dagSeed now records what each DAG was generated from and both
Block_EnsureAutolykos2Dag and Block_PowHashHeavy compare it.

== Bug found in testing: double free on the failed-apply path ==

SIGABRT in the allocator: free_tiny_botch -> DynArr_destroy -> Block_Destroy ->
Chain_FreeBlockArray -> Chain_ReplaceBranch.

DynArr_push_back stores the struct BY VALUE, so the chain's element and the
caller's block_t share one transactions pointer. Three places free that array
through the chain's copy -- Chain_ClearBlocks, Chain_RollbackToHeightLocked and
Chain_SaveToFile -- and each NULLs only the chain's side, leaving any caller
wrapper dangling. Whether a caller then had to use free() or Block_Destroy() was
a convention carried in comments at every call site plus a consumed-count passed
into Chain_FreeBlockArray. Chain_ReplaceBranch reset that count to 0 after
rolling back a failed apply, which told the cleanup to Block_Destroy exactly the
blocks whose arrays the rollback had just freed.

Rather than fix the count, the aliasing is now safe by construction:
Chain_AddBlockLocked clears the CALLER's transactions pointer immediately after
the push. Since DynArr_destroy(NULL) is a no-op, free(wrapper) and
Block_Destroy(wrapper) become equivalent and both safe regardless of what later
frees the chain's copy. The consumed-count parameter and both counters are gone
-- the thing that could be got wrong no longer exists -- and all call sites are
unified on Block_Destroy.

Placement is deliberate: immediately after the push, not at the end on success.
The ledger pass can fail with the block already in the chain, returning false to
a caller that destroys its wrapper on failure -- OrphanPool_ExtendTip does
exactly that, a third live instance not yet triggered.

Two follow-ons the refactor forced, both improvements anyway: MineAndAppendBlock
read the coinbase for its log line after the add (hoisted above it), and the
success log printed the caller's block rather than the chain's copy.

== Also ==

The deferral line is rate-limited. The maintenance thread retries pooled
branches once a second and elapsed only changes when something moves, so it
printed an identical line every second -- forever, on a node that is not mining.
It now reports each distinct situation once.

== Verification ==

Synthetic fork, node A 5 deep, node B ~60 ahead, EPOCH_LENGTH=8 so the branch
crosses three epoch boundaries:

  branchLead climbs 8 -> 16 -> 28 -> 34, crosses penalty(5)=42
  Adopted competing branch of 50 block(s) at fork height 20
  sync complete: localHeight=70
  Chain OK

Repeated under AddressSanitizer: adopted 47 blocks, 0 ASan errors on both nodes.
This matters because the refactor rewrites the exact cleanup path the SIGABRT
came from, and a double free that no longer aborts would otherwise pass silently.

Unit suites pass, including a new assertion "heavy path refuses a DAG built from
a different seed" -- the direct regression for the stale-DAG bug.

Harness note: each node needs its OWN wallet. With a shared one both pay the
same coinbase address, produce identical merkle roots, and at easy difficulty
mine byte-identical blocks -- the fork test silently became a catch-up test.

== Still untested ==

The restore-after-failed-apply path is no longer naturally reachable now that
the two bugs above are fixed, so it needs deliberate corruption to exercise.
Test B (branch only slightly ahead must still DEFER), test C (sync force), and a
TSan pass over the changed paths are outstanding.
This commit is contained in:
2026-08-02 19:36:09 +02:00
parent 01c44731ef
commit 5eaf0b699c
8 changed files with 349 additions and 115 deletions
+23 -3
View File
@@ -53,9 +53,11 @@ void Block_RemoveTransaction(block_t* block, uint8_t* txHash);
* DAG was last built for).
**/
bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8_t seed32[32]);
// Fails rather than answering from a DAG built for a different epoch or size, so it can never
// silently hash against the wrong lanes.
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes, uint8_t outHash[32]);
// Fails rather than answering from a DAG built for a different epoch, size OR SEED, so it can
// never silently hash against the wrong lanes. The seed matters because a reorg changes it while
// leaving the epoch index and size unchanged.
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]);
bool Block_PowHashLight(const block_t* block, size_t dagBytes, const uint8_t seed32[32], uint8_t outHash[32]);
// PoW check against explicitly supplied epoch parameters, for callers that resolve them once and
@@ -72,6 +74,24 @@ bool Block_HasValidVote(const block_t* block);
bool Block_AllTransactionsValid(const block_t* block);
bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinbaseAmount, uint64_t* outTotalFees);
/**
* Self-contained validity: merkle root, transactions, vote encoding, non-empty. Needs no chain, so
* it is meaningful for ANY block, including one on a branch we do not have.
*
* This is what the receive path checks. Proof of work is deliberately NOT checked there, because
* PoW is only meaningful relative to the branch a block belongs to: the epoch seed is the last
* block of the previous epoch on ITS OWN branch. Validating a competing branch's block against our
* epoch seed does not merely fail to resolve -- when the two chains diverge before the boundary it
* resolves to the WRONG seed and rejects a perfectly valid block, which made any fork spanning an
* epoch boundary impossible to assemble.
*
* Chain_AddBlock verifies proof of work at the moment a block joins the chain, where the branch
* context is real. That, not the receive path, is what enforces the invariant.
**/
bool Block_HasValidStructure(const block_t* block);
// Full check including chain-relative PoW. Only meaningful for a block that extends `chain`.
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain);
void Block_ShutdownPowContext(void);
void Block_Destroy(block_t* block);
+8 -1
View File
@@ -65,6 +65,12 @@ bool Chain_RollbackToHeight(blockchain_t* chain, size_t height);
* see the comment in the implementation. The initial-block-download exemption is decided inside,
* from local state only, so no caller can switch the penalty off.
*
* `bypassPenalty` skips the delay check ONLY. It exists for an explicit operator action (`sync
* force`) on a node whose chain is known to be the wrong one -- the penalty is served by local
* chain growth, so a node that is neither mining nor stale enough to count as catching up cannot
* clear it on its own. It must never be reachable from anything a peer says; work comparison,
* linkage and atomicity are still enforced, so this cannot adopt a branch that is not heavier.
*
* On any failure the original chain, balance sheet, supply and reward are restored and false is
* returned. The caller keeps ownership of `newBlocks` in every case: the chain applies copies.
**/
@@ -72,7 +78,8 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
size_t forkHeight,
block_t** newBlocks,
size_t count,
uint64_t observedAtTipHeight);
uint64_t observedAtTipHeight,
bool bypassPenalty);
// True when this node is catching up rather than following the tip (empty chain, or a median
// block time far in the past). Used to exempt initial sync from the reorg penalty.
+11
View File
@@ -22,6 +22,17 @@ void OrphanPool_Insert(block_t* block, uint64_t height, uint64_t observedAtTipHe
// Returns the number of blocks successfully attached.
size_t OrphanPool_AttemptAttach(blockchain_t* chain);
/**
* As OrphanPool_AttemptAttach, but skips the reorg delay penalty when `bypassPenalty` is set.
*
* Reserved for an explicit operator action (`sync force`). The penalty is served by local chain
* growth, so a node that is neither mining nor stale enough to count as catching up can never
* clear it by itself; this is the manual way out for an operator who knows their branch is the
* wrong one. Work comparison and linkage still apply, so it cannot adopt a lighter branch, and
* nothing a peer sends can reach it.
**/
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty);
// True if a block with this hash is already pooled.
bool OrphanPool_Contains(const uint8_t blockHash[32]);
+37 -17
View File
@@ -16,6 +16,10 @@
static Autolykos2Context* g_autolykos2Ctx = NULL;
static pthread_mutex_t g_powCtxLock = PTHREAD_MUTEX_INITIALIZER;
static uint64_t g_dagEpoch = 0;
// The seed the current DAG was generated from. Matching on epoch index and size is NOT enough: a
// reorg replaces the block an epoch's seed is derived from while leaving the epoch index and size
// unchanged, so a stale DAG would still look current and silently hash against the wrong lanes.
static uint8_t g_dagSeed[32];
static bool g_dagReady = false;
// Caller must hold `g_powCtxLock`.
@@ -50,9 +54,11 @@ bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8
pthread_mutex_lock(&g_powCtxLock);
// Already built for this epoch at this size: generation is seconds of work, so never redo it.
// Already built from exactly this seed at this size: generation is seconds of work, never redo
// it. The seed has to be part of the test -- see g_dagSeed.
if (g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes) {
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0) {
pthread_mutex_unlock(&g_powCtxLock);
return true;
}
@@ -70,6 +76,7 @@ bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8
const bool ok = Autolykos2_DagAllocate(ctx, dagBytes) && Autolykos2_DagGenerate(ctx, seed32);
if (ok) {
g_dagEpoch = epochIndex;
memcpy(g_dagSeed, seed32, 32);
g_dagReady = true;
}
@@ -77,17 +84,22 @@ bool Block_EnsureAutolykos2Dag(uint64_t epochIndex, size_t dagBytes, const uint8
return ok;
}
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes, uint8_t outHash[32]) {
if (!block || !outHash) {
bool Block_PowHashHeavy(const block_t* block, uint64_t epochIndex, size_t dagBytes,
const uint8_t seed32[32], uint8_t outHash[32]) {
if (!block || !seed32 || !outHash) {
return false;
}
pthread_mutex_lock(&g_powCtxLock);
// Checking the epoch and size here, rather than trusting the caller to have built the right
// DAG, is what makes this impossible to misuse: an unbuilt or stale DAG yields false and the
// caller falls back to deriving the lanes from the seed.
// Verifying the SEED here, not just the epoch and size, is what makes this impossible to
// misuse. A reorg changes the block an epoch's seed is derived from while the epoch index and
// size stay put, so an epoch+size check alone happily accepts a DAG built from the pre-reorg
// seed and returns a hash for the wrong lanes -- which shows up as a valid block failing PoW
// while a branch is being applied. A mismatch yields false and the caller derives the lanes
// from the seed instead.
const bool usable = g_dagReady && g_autolykos2Ctx && g_dagEpoch == epochIndex &&
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes;
Autolykos2_DagSize(g_autolykos2Ctx) == dagBytes &&
memcmp(g_dagSeed, seed32, 32) == 0;
const bool ok = usable &&
Autolykos2_Hash(
g_autolykos2Ctx,
@@ -254,7 +266,7 @@ bool Block_HasValidProofOfWorkWithParams(const block_t* block, uint64_t epochInd
// from the epoch seed. The two produce identical hashes, so which one runs is invisible to
// consensus; only speed differs.
uint8_t hash[32];
if (!Block_PowHashHeavy(block, epochIndex, dagBytes, hash) &&
if (!Block_PowHashHeavy(block, epochIndex, dagBytes, seed32, hash) &&
!Block_PowHashLight(block, dagBytes, seed32, hash)) {
// Fail CLOSED. This used to hand back a zeroed hash on any failure and compare that to the
// target -- and zero is below every target, so a DAG that was missing, mis-sized or failed
@@ -383,16 +395,24 @@ bool Block_ValidateCoinbaseAndFees(const block_t* block, uint64_t expectedCoinba
return true;
}
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain) {
bool merkleValid = false;
uint8_t calculatedMerkleRoot[32];
if (block && block->transactions) {
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
merkleValid = (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) == 0);
bool Block_HasValidStructure(const block_t* block) {
if (!block || !block->transactions) {
return false;
}
return Block_HasValidVote(block) && Block_HasValidProofOfWork(block, chain) &&
Block_AllTransactionsValid(block) && DynArr_size(block->transactions) > 0 && merkleValid;
uint8_t calculatedMerkleRoot[32];
Block_CalculateMerkleRoot(block, calculatedMerkleRoot);
if (memcmp(calculatedMerkleRoot, block->header.merkleRoot, 32) != 0) {
return false;
}
return Block_HasValidVote(block) &&
Block_AllTransactionsValid(block) &&
DynArr_size(block->transactions) > 0;
}
bool Block_IsFullyValid(const block_t* block, blockchain_t* chain) {
return Block_HasValidStructure(block) && Block_HasValidProofOfWork(block, chain);
}
void Block_Destroy(block_t* block) {
+146 -30
View File
@@ -10,6 +10,11 @@
uint64_t currentBlockHeight = 0;
// Defined near the DAG helpers at the bottom; needed early by Chain_AddBlockLocked, which verifies
// proof of work while already holding chainLock for writing.
static bool Chain_DagParamsForHeightLocked(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]);
static bool EnsureDirectoryExists(const char* dirpath) {
if (!dirpath || dirpath[0] == '\0') {
return false;
@@ -263,6 +268,7 @@ void Chain_Destroy(blockchain_t* chain) {
**/
static bool Chain_AddBlockLocked(blockchain_t* chain, block_t* block) {
bool ok = true;
block_t* stored = NULL; // the chain's own copy; the caller's `block` loses its transactions below
if (!chain || !block || !chain->blocks || !block->transactions) {
return false;
@@ -295,6 +301,34 @@ static bool Chain_AddBlockLocked(blockchain_t* chain, block_t* block) {
// Ensure the block was mined at the difficulty this chain requires at that height. Without this
// a peer whose difficulty went stale (or a malicious one) can hand us a block mined at an
// easier target, which Block_HasValidProofOfWork accepts because it checks the header's own value.
/**
* Proof of work is verified HERE, at the moment a block joins the chain, and not only on the
* receive path.
*
* A block on a competing branch cannot always be PoW-checked when it arrives: its epoch seed is
* the last block of the previous epoch ON ITS OWN BRANCH, which may still be sitting in the
* orphan pool rather than in our chain. Those blocks are pooled with PoW deferred, so this is
* the checkpoint that keeps the real invariant -- nothing enters the chain without verified
* work. By the time a branch is applied the rollback has put its ancestors in place, so the
* epoch parameters always resolve here.
**/
{
size_t dagBytes = 0;
uint8_t dagSeed[32];
if (!Chain_DagParamsForHeightLocked(chain, block->header.blockNumber, &dagBytes, dagSeed)) {
printf("Chain_AddBlock: validation failed: blockIndex=%zu cannot resolve epoch DAG parameters\n",
expectedIndex);
return false;
}
const uint64_t epochIndex = block->header.blockNumber / (uint64_t)EPOCH_LENGTH;
if (!Block_HasValidProofOfWorkWithParams(block, epochIndex, dagBytes, dagSeed)) {
printf("Chain_AddBlock: validation failed: blockIndex=%zu proof of work is invalid\n",
expectedIndex);
return false;
}
}
uint32_t expectedTarget = Chain_GetTargetForHeight(chain, (uint64_t)expectedIndex);
if (block->header.difficultyTarget != expectedTarget) {
printf("Chain_AddBlock: validation failed: blockIndex=%zu expectedDifficulty=%#x observedDifficulty=%#x\n",
@@ -393,9 +427,30 @@ static bool Chain_AddBlockLocked(blockchain_t* chain, block_t* block) {
// Push the block only after validation succeeds.
block_t* blk = (block_t*)DynArr_push_back(chain->blocks, block);
if (!blk) { ok = false; break; }
stored = blk;
chain->size++;
currentBlockHeight = (uint64_t)(chain->size - 1);
/**
* The chain now owns the transaction array, so clear the CALLER's pointer to it.
*
* DynArr_push_back stores the struct by value, which leaves the chain's element and the
* caller's block_t sharing one `transactions` pointer. Three separate places later free
* that array through the chain's copy -- Chain_ClearBlocks, Chain_RollbackToHeightLocked
* and Chain_SaveToFile -- and each of them NULLs only the chain's side, leaving the
* caller holding a dangling pointer. Whether a caller must then use free() or
* Block_Destroy() was a convention carried in comments at every call site plus a
* consumed-count passed around; getting it wrong aborted in the allocator.
*
* Clearing it here makes the rule structural: free(wrapper) and Block_Destroy(wrapper)
* are now equivalent and both safe, because DynArr_destroy(NULL) is a no-op.
*
* This runs right after the push and NOT at the end on success, deliberately. If the
* ledger pass below fails we return false with the block still in the chain, so a caller
* that destroys its wrapper on failure would otherwise free the chain's array.
**/
block->transactions = NULL;
// Second pass: apply the ledger changes.
if (blk->transactions) {
txCount = DynArr_size(blk->transactions);
@@ -464,7 +519,7 @@ static bool Chain_AddBlockLocked(blockchain_t* chain, block_t* block) {
if (ok) {
printf("Added new block to chain:\n");
Block_ShortPrint(block);
Block_ShortPrint(stored ? stored : block); // stored still has the transactions; `block` no longer does
}
return ok;
@@ -795,20 +850,21 @@ static bool Chain_BranchIsLinkedLocked(blockchain_t* chain, size_t forkHeight, b
return true;
}
static void Chain_FreeBlockArray(block_t** blocks, size_t count, size_t consumedByChain) {
/**
* Free an array of blocks and the array itself.
*
* No consumed-count is needed: Chain_AddBlockLocked clears the caller's `transactions` pointer when
* it takes ownership, so Block_Destroy is correct whether or not a given block reached the chain.
* The count used to be threaded through here, and getting it wrong -- resetting it after a rollback
* had already freed the aliased arrays -- is what aborted in the allocator.
**/
static void Chain_FreeBlockArray(block_t** blocks, size_t count) {
if (!blocks) {
return;
}
for (size_t i = 0; i < count; ++i) {
if (!blocks[i]) {
continue;
}
if (i < consumedByChain) {
// Chain_AddBlockLocked shallow-copies the struct, so the chain owns the transactions
// now. Free only our wrapper -- Block_Destroy would take the chain's array with it.
free(blocks[i]);
} else {
if (blocks[i]) {
Block_Destroy(blocks[i]);
}
}
@@ -824,7 +880,8 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
size_t forkHeight,
block_t** newBlocks,
size_t count,
uint64_t observedAtTipHeight) {
uint64_t observedAtTipHeight,
bool bypassPenalty) {
if (!chain || !chain->blocks || !newBlocks || count == 0 || forkHeight == 0) {
return false;
}
@@ -835,9 +892,8 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
bool ok = false;
block_t** snapshot = NULL; // deep copies of the blocks we are replacing
size_t snapshotCount = 0;
size_t snapshotConsumed = 0;
block_t** candidate = NULL; // deep copies of the branch we are applying
size_t candidateConsumed = 0;
size_t candidateApplied = 0; // how far the apply got, for the log line below
do {
const size_t tipCount = DynArr_size(chain->blocks);
@@ -862,18 +918,64 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
// in by a caller: a peer claiming a huge height must not be able to switch the penalty off.
const bool inInitialBlockDownload = Chain_IsInitialBlockDownload(chain);
const uint64_t tipHeight = tipCount > 0 ? (uint64_t)(tipCount - 1) : 0ULL;
if (!inInitialBlockDownload && tipCount > forkHeight) {
if (bypassPenalty && tipCount > forkHeight) {
// Operator override, never reachable from anything a peer sends. Announced loudly
// because it is the one place the delay protection is deliberately not applied.
printf("Chain_ReplaceBranch: reorg penalty BYPASSED at height %zu by operator request\n",
forkHeight);
}
if (!bypassPenalty && !inInitialBlockDownload && tipCount > forkHeight) {
const uint64_t observedTip = observedAtTipHeight > tipHeight ? tipHeight : observedAtTipHeight;
const uint64_t depth = observedTip >= (uint64_t)forkHeight
? (observedTip - (uint64_t)forkHeight + 1ULL)
: 1ULL;
const uint64_t penalty = FetchScheduler_ComputeReorgPenaltyBlocks(depth);
const uint64_t elapsed = tipHeight >= observedTip ? (tipHeight - observedTip) : 0ULL;
/**
* The delay can be served by EITHER side making progress.
*
* Measuring it against our own chain growth alone assumed we are an active participant
* whose tip advances. A node that is behind is not: its tip is frozen precisely because
* it is rejecting the branch, so `elapsed` stays 0 forever and it can never rejoin. That
* is not a reorg contest at all -- being 500 blocks behind is being behind, and
* refusing to adopt protects nothing while the node falls further back every hour.
*
* A branch that already extends `penalty` blocks past our tip has demonstrated exactly
* what the delay asks for -- that much sustained public work -- and every one of those
* blocks carries PoW we validated ourselves. Requiring us to independently produce the
* same amount is demanding the same proof twice. Counting only VALIDATED blocks is what
* keeps this safe: a peer's advertised height is not evidence and never reaches here.
*
* The rule degrades correctly in both directions. A mining node's tip advances, so an
* attacker has to outpace it by `penalty` blocks. A node that is merely observing
* follows the heaviest chain, which is what an observer should do.
**/
const uint64_t localGrowth = tipHeight >= observedTip ? (tipHeight - observedTip) : 0ULL;
const uint64_t candidateTip = (uint64_t)forkHeight + (uint64_t)count - 1ULL;
const uint64_t branchLead = candidateTip > tipHeight ? (candidateTip - tipHeight) : 0ULL;
const uint64_t elapsed = localGrowth > branchLead ? localGrowth : branchLead;
if (elapsed < penalty) {
printf("Chain_ReplaceBranch: deferring reorg at height %zu: depth=%" PRIu64
" penalty=%" PRIu64 " elapsed=%" PRIu64 "\n",
forkHeight, depth, penalty, elapsed);
// The maintenance thread retries pooled branches once a second, and `elapsed` can
// only change when our own tip moves -- so without this the same line is printed
// every second for as long as the branch stays deferred, which on a node that is
// not mining is forever. Report each distinct deferral once, and again whenever the
// situation actually changes. Guarded by chainLock (held for writing here).
static size_t lastDeferredFork = SIZE_MAX;
static uint64_t lastDeferredTip = UINT64_MAX;
static uint64_t lastDeferredPenalty = UINT64_MAX;
static uint64_t lastDeferredElapsed = UINT64_MAX;
if (forkHeight != lastDeferredFork || tipHeight != lastDeferredTip ||
penalty != lastDeferredPenalty || elapsed != lastDeferredElapsed) {
printf("Chain_ReplaceBranch: deferring reorg at height %zu: depth=%" PRIu64
" penalty=%" PRIu64 " elapsed=%" PRIu64
" (localGrowth=%" PRIu64 " branchLead=%" PRIu64 ")\n",
forkHeight, depth, penalty, elapsed, localGrowth, branchLead);
lastDeferredFork = forkHeight;
lastDeferredTip = tipHeight;
lastDeferredPenalty = penalty;
lastDeferredElapsed = elapsed;
}
break;
}
}
@@ -966,7 +1068,7 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
applied = false;
break;
}
candidateConsumed++;
candidateApplied++;
}
if (applied) {
@@ -976,14 +1078,13 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
// Undo: drop whatever of the candidate branch made it in and put the original blocks back.
printf("Chain_ReplaceBranch: candidate branch failed to apply at height %zu; restoring previous chain\n",
forkHeight + candidateConsumed);
forkHeight + candidateApplied);
if (!Chain_RollbackToHeightLocked(chain, forkHeight)) {
fprintf(stderr, "Chain_ReplaceBranch: FAILED to roll back after a failed apply; chain is truncated to %zu\n",
forkHeight);
break;
}
candidateConsumed = 0; // the rollback freed the copies' transactions
for (size_t i = 0; i < snapshotCount; ++i) {
if (!Chain_AddBlockLocked(chain, snapshot[i])) {
@@ -991,12 +1092,11 @@ bool Chain_ReplaceBranch(blockchain_t* chain,
forkHeight + i, forkHeight + i);
break;
}
snapshotConsumed++;
}
} while (0);
Chain_FreeBlockArray(snapshot, snapshotCount, snapshotConsumed);
Chain_FreeBlockArray(candidate, candidate ? count : 0, candidateConsumed);
Chain_FreeBlockArray(snapshot, snapshotCount);
Chain_FreeBlockArray(candidate, candidate ? count : 0);
pthread_mutex_unlock(&balanceSheetLock);
pthread_rwlock_unlock(&chainLock);
@@ -1879,8 +1979,15 @@ static bool Chain_EpochDagSeedForHeightLocked(blockchain_t* chain, uint64_t bloc
return true;
}
bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]) {
/**
* As Chain_DagParamsForHeight, but assumes `chainLock` is already held (read OR write).
*
* Needed because Chain_AddBlockLocked verifies proof of work while holding the write lock, and
* chainLock is not recursive -- taking it for reading there deadlocks the moment another thread
* queues for the write lock.
**/
static bool Chain_DagParamsForHeightLocked(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]) {
if (!chain || !chain->blocks || !outDagBytes || !outSeed) {
return false;
}
@@ -1890,8 +1997,6 @@ bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
uint64_t bytes = 0;
bool ok = false;
pthread_rwlock_rdlock(&chainLock);
// Lock order is chainLock -> dagCacheLock, everywhere. Nothing under dagCacheLock calls back
// into chain.c, so this pair cannot deadlock.
pthread_mutex_lock(&chain->dagCacheLock);
@@ -1905,8 +2010,6 @@ bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
ok = Chain_EpochDagSeedForHeightLocked(chain, blockHeight, outSeed);
}
pthread_rwlock_unlock(&chainLock);
if (!ok) {
return false;
}
@@ -1921,6 +2024,19 @@ bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
return true;
}
bool Chain_DagParamsForHeight(blockchain_t* chain, uint64_t blockHeight,
size_t* outDagBytes, uint8_t outSeed[32]) {
if (!chain || !chain->blocks || !outDagBytes || !outSeed) {
return false;
}
pthread_rwlock_rdlock(&chainLock);
const bool ok = Chain_DagParamsForHeightLocked(chain, blockHeight, outDagBytes, outSeed);
pthread_rwlock_unlock(&chainLock);
return ok;
}
void Chain_OnTipAdvanced(blockchain_t* chain) {
if (!chain || !chain->blocks) {
return;
+98 -50
View File
@@ -478,11 +478,8 @@ static bool MineAndAppendBlock(blockchain_t* chain,
return false;
}
if (!Chain_AddBlock(chain, block)) {
fprintf(stderr, "failed to append block to chain\n");
return false;
}
// Read the coinbase BEFORE handing the block to the chain. Chain_AddBlock takes ownership of
// the transaction array and clears our pointer to it, so this has to happen first.
uint64_t coinbaseAmount = 0;
if (block->transactions && DynArr_size(block->transactions) > 0) {
signed_transaction_t* firstTx = (signed_transaction_t*)DynArr_at(block->transactions, 0);
@@ -491,6 +488,11 @@ static bool MineAndAppendBlock(blockchain_t* chain,
}
}
if (!Chain_AddBlock(chain, block)) {
fprintf(stderr, "failed to append block to chain\n");
return false;
}
/* Debug proof removed: miner printed proof that coinbase == baseReward + totalFees during debugging. */
// After successfully appending a block, attempt to attach any orphans.
@@ -922,7 +924,7 @@ int main(int argc, char* argv[]) {
char supplyStr[80];
Uint256ToDecimal(&currentSupply, supplyStr, sizeof(supplyStr));
printf("Current chain has %zu blocks, total supply %s\n", Chain_Size(chain), supplyStr);
printf("Commands: mine <x>, send <address> <amount> [fee], txpooldetail <txhash>, balance [address], connect <ipv4> [port], peers, sync (requires nodes), flushchain, fullverify, blockdetail <block number>, wipechain, genaddr, exit\n");
printf("Commands: mine <x>, send <address> <amount> [fee], txpooldetail <txhash>, balance [address], connect <ipv4> [port], peers, sync [force] (requires nodes), dagvote <grow|hold|down>, flushchain, fullverify, blockdetail <block number>, wipechain, genaddr, exit\n");
char line[1024];
while (true) {
@@ -1025,7 +1027,7 @@ int main(int argc, char* argv[]) {
break;
}
free(block); // Chain stores block by value and owns copied transaction array.
Block_Destroy(block); // Chain_AddBlock already took the transaction array.
// Broadcast newly mined block to outbound peers
if (node) {
@@ -1132,7 +1134,7 @@ int main(int argc, char* argv[]) {
FlushChainAndSheet(chain, chainDataDir, currentSupply, currentReward);
free(block);
Block_Destroy(block); // the chain took the transaction array; this frees the wrapper
if (node) {
Node_BroadcastChainRange(node, Chain_Size(chain) - 1, NULL);
}
@@ -1162,6 +1164,24 @@ int main(int argc, char* argv[]) {
continue;
}
// `sync force` skips the reorg delay penalty for this sync only. The penalty is served
// by local chain growth, so a node that is neither mining nor stale enough to count as
// catching up cannot clear it on its own -- this is the operator's way out when they
// know their branch is the wrong one. Work comparison and linkage still apply.
bool forceSync = false;
{
const char* syncArg = strtok(NULL, " \t");
if (syncArg && strcmp(syncArg, "force") == 0) {
forceSync = true;
} else if (syncArg) {
printf("usage: sync [force]\n");
continue;
}
}
if (forceSync) {
printf("sync force: the reorg delay penalty will be skipped for this sync\n");
}
// Choose the best outbound peer by advertised height
tcp_connection_t* peerConn = NULL;
uint64_t peerHeight = 0;
@@ -1174,6 +1194,12 @@ int main(int argc, char* argv[]) {
bool madeProgressOverall = false;
int forkProbes = 0;
// Fork state for this sync. The backward walk runs once; after that the window keeps
// marching forward so the competing branch accumulates in the pool, which is what lets
// its length past our tip satisfy the reorg delay.
bool forkPointLocated = false;
uint64_t pooledSinceAttach = 0;
// Drop receipts left over from an earlier sync, so a stale one cannot be mistaken for
// an answer to a request this run has not sent yet.
Node_ResetBlockDeliveries();
@@ -1285,7 +1311,7 @@ int main(int argc, char* argv[]) {
printf("Divergence at height %" PRIu64 "; probing for the fork point\n", h);
RequestForkWindow(node, peerConn, h);
size_t reattached = OrphanPool_AttemptAttach(chain);
size_t reattached = OrphanPool_AttemptAttachForced(chain, forceSync);
if (reattached > 0) {
printf("Reorg attached %zu block(s) from the peer's branch\n", reattached);
} else {
@@ -1334,51 +1360,62 @@ int main(int argc, char* argv[]) {
// and made the peer re-serve the whole chain several times over.
node_delivery_status_t deliveryStatus = NODE_DELIVERY_REJECTED;
if (Node_TakeBlockDelivery(h, &deliveryStatus) && deliveryStatus != NODE_DELIVERY_APPENDED) {
if (forkProbes >= MAX_FORK_PROBE_ROUNDS) {
printf("block %" PRIu64 " is on a branch we cannot join after %d probe(s); "
"giving up on this peer\n", h, forkProbes);
inFlight = 0;
nextReq = end; // stop refilling the window; this peer is unreachable by extension
break;
}
// Locate the fork point ONCE. After that the branch just needs to keep
// accumulating: the delay is satisfied by the branch extending past our tip
// (see Chain_ReplaceBranch), so the window must march FORWARD pooling
// blocks. Resetting it to our tip on every forked delivery -- which is what
// this used to do -- re-requested the same eight heights forever, so the
// pool never grew past the window size and a node that was far behind could
// never accumulate enough of the branch to adopt it.
if (!forkPointLocated) {
forkProbes++;
printf("block %" PRIu64 " arrived but does not extend our chain; "
"probing for the fork point\n", h);
forkProbes++;
printf("block %" PRIu64 " arrived but does not extend our chain; "
"probing for the fork point (round %d/%d)\n",
h, forkProbes, MAX_FORK_PROBE_ROUNDS);
// Pull a window BELOW the divergence so the pool can walk prevHash back to
// the common ancestor. Without this the fork point is never requested at
// all, because the window only ever moves forward from our own tip.
const bool foundCommon = RequestForkWindow(node, peerConn, h);
size_t reattached = foundCommon ? OrphanPool_AttemptAttach(chain) : 0;
if (reattached > 0) {
printf("Reorg attached %zu block(s) from the peer's branch\n", reattached);
forkProbes = 0; // real progress; allow probing again if it forks further on
} else if (!foundCommon) {
printf("No shared block found with this peer; its branch cannot be linked to ours\n");
} else {
// Deliberately not phrased as a failure: the branch stays pooled and the
// 1Hz maintenance thread retries it, which is usually what completes a
// reorg whose blocks were still arriving when this pass ran.
printf("Reorg not completed on this pass; branch stays pooled for retry\n");
}
// Restart the window against whatever our tip is now.
nextReq = Chain_Size(chain);
inFlight = 0;
if (Chain_Size(chain) > 0) {
block_t* tip = NULL;
if (Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &tip) && tip) {
Block_CalculateHash(tip, expectedPrevHash);
Block_Destroy(tip);
forkPointLocated = RequestForkWindow(node, peerConn, h);
if (!forkPointLocated) {
printf("No shared block found with this peer within %" PRIu64
" blocks; its branch cannot be linked to ours\n", REORG_FETCH_DEPTH);
inFlight = 0;
nextReq = end; // this peer is unreachable by extension
break;
}
} else {
memset(expectedPrevHash, 0, sizeof(expectedPrevHash));
}
// Delivered, so drop it from the in-flight set and let the window advance.
for (int j = i; j < inFlight - 1; ++j) {
requestedHeights[j] = requestedHeights[j + 1];
retryCount[j] = retryCount[j + 1];
sentAtMs[j] = sentAtMs[j + 1];
}
inFlight--;
pooledSinceAttach++;
progressed = true;
// Retry adoption as the branch grows, rather than once per probe. Each
// attempt walks the pool, so do it per window-full instead of per block.
if (pooledSinceAttach >= (uint64_t)maxInFlight) {
pooledSinceAttach = 0;
size_t reattached = OrphanPool_AttemptAttachForced(chain, forceSync);
if (reattached > 0) {
printf("Reorg attached %zu block(s) from the peer's branch\n", reattached);
// The chain moved; realign the window and the expected parent hash.
nextReq = Chain_Size(chain);
inFlight = 0;
forkPointLocated = false; // a further divergence would need a new walk
if (Chain_Size(chain) > 0) {
block_t* tip = NULL;
if (Chain_GetBlockCopy(chain, Chain_Size(chain) - 1, &tip) && tip) {
Block_CalculateHash(tip, expectedPrevHash);
Block_Destroy(tip);
}
} else {
memset(expectedPrevHash, 0, sizeof(expectedPrevHash));
}
}
}
break;
}
@@ -1411,6 +1448,17 @@ int main(int argc, char* argv[]) {
}
}
// The window drained. Anything pooled since the last attempt has not been tried yet --
// without this a branch whose final partial batch never reached the retry threshold
// would sit in the pool unadopted until the maintenance thread happened to pick it up.
if (pooledSinceAttach > 0) {
pooledSinceAttach = 0;
size_t reattached = OrphanPool_AttemptAttachForced(chain, forceSync);
if (reattached > 0) {
printf("Reorg attached %zu block(s) from the peer's branch\n", reattached);
}
}
// After the window completes, check progress and possibly refresh peer height.
// This flag tracks THIS iteration only: it used to be set once and never cleared, so
// after a single productive pass the "no progress -> stop" guard below could never fire
@@ -1445,7 +1493,7 @@ int main(int argc, char* argv[]) {
peerHeight, newLocal);
const bool foundCommon = RequestForkWindow(node, peerConn, newLocal);
size_t attached = foundCommon ? OrphanPool_AttemptAttach(chain) : 0;
size_t attached = foundCommon ? OrphanPool_AttemptAttachForced(chain, forceSync) : 0;
if (attached > 0) {
printf("Fork probe adopted %zu block(s) from the peer's branch\n", attached);
continue;
+16 -9
View File
@@ -533,8 +533,15 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
return NODE_BLOCK_REJECTED;
}
// Validate block
if (!Block_IsFullyValid(blk, currentChain)) {
// Only the self-contained checks run here. Proof of work is verified by Chain_AddBlock, at the
// point a block actually joins the chain.
//
// PoW cannot be judged here because it is relative to the branch the block belongs to: the
// epoch seed is the last block of the previous epoch on ITS branch. For a block on a competing
// branch our chain gives the WRONG seed whenever the two diverge before that boundary, so
// checking it here rejected perfectly valid blocks and made any fork spanning an epoch boundary
// impossible to assemble. Deferring costs at most a slot in a pool that is already capped.
if (!Block_HasValidStructure(blk)) {
printf("Rejected BLOCK_DATA at height %" PRIu64 " during validation\n", blockHeight);
DynArr_destroy(blk->transactions);
free(blk);
@@ -607,12 +614,11 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
}
if (!Chain_AddBlock(currentChain, blk)) {
// Chain_AddBlock failed; cleanup
// Chain_AddBlock failed; cleanup. Safe either way: if it failed before taking the block we
// still own the transactions, and if it failed after (the ledger pass can fail with the
// block already pushed) our pointer to them was cleared, so this frees only the wrapper.
printf("Rejected BLOCK_DATA at height %" PRIu64 " during chain add\n", blockHeight);
if (blk->transactions) {
DynArr_destroy(blk->transactions);
}
free(blk);
Block_Destroy(blk);
return NODE_BLOCK_REJECTED;
}
@@ -625,8 +631,9 @@ static node_block_accept_result_t Node_ParseAndAcceptBlock(const unsigned char*
BalanceSheet_SaveToFile(chainDataDir);
}
// Chain_AddBlock copied the block into the chain; free our temporary wrapper but do NOT destroy transactions (they are freed by Chain_SaveToFile when persisted)
free(blk);
// Chain_AddBlock took ownership of the transaction array and cleared our pointer to it, so
// destroying the wrapper here frees only the wrapper.
Block_Destroy(blk);
// Attempt to attach any orphans that may now have their parents present.
size_t attached = OrphanPool_AttemptAttach(currentChain);
if (attached > 0) {
+10 -5
View File
@@ -441,8 +441,8 @@ static size_t OrphanPool_ExtendTip(blockchain_t* chain) {
continue;
}
// The chain took over the copy's transaction array; free only our wrapper.
free(candidate);
// Chain_AddBlock took ownership of the transaction array and cleared our pointer to it.
Block_Destroy(candidate);
pthread_mutex_lock(&g_orphanLock);
block_t* taken = OrphanPool_TakeByHashLocked(candidateHash);
@@ -461,7 +461,7 @@ static size_t OrphanPool_ExtendTip(blockchain_t* chain) {
* Look for a competing branch that forks below our tip and is worth adopting.
* The work comparison, the reorg penalty and the atomicity all live in Chain_ReplaceBranch.
**/
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain) {
static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain, bool bypassPenalty) {
const size_t chainSize = Chain_Size(chain);
if (chainSize == 0) {
return 0;
@@ -516,7 +516,8 @@ static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain) {
bool adopted = false;
if (copiedAll) {
adopted = Chain_ReplaceBranch(chain, forkHeight, branchCopies, branchCount, observedAtTipHeight);
adopted = Chain_ReplaceBranch(chain, forkHeight, branchCopies, branchCount, observedAtTipHeight,
bypassPenalty);
}
for (size_t i = 0; i < branchCount; ++i) {
@@ -548,6 +549,10 @@ static size_t OrphanPool_TryAdoptBranch(blockchain_t* chain) {
}
size_t OrphanPool_AttemptAttach(blockchain_t* chain) {
return OrphanPool_AttemptAttachForced(chain, false);
}
size_t OrphanPool_AttemptAttachForced(blockchain_t* chain, bool bypassPenalty) {
if (!chain) {
return 0;
}
@@ -567,7 +572,7 @@ size_t OrphanPool_AttemptAttach(blockchain_t* chain) {
size_t extended = OrphanPool_ExtendTip(chain);
attached += extended;
size_t adopted = OrphanPool_TryAdoptBranch(chain);
size_t adopted = OrphanPool_TryAdoptBranch(chain, bypassPenalty);
attached += adopted;
if (extended == 0 && adopted == 0) {