Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expression signing and verification migrated to Rust #452

Merged
merged 31 commits into from
Feb 28, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f3fbbc9
Add signatures.rs
lucksus Feb 20, 2024
3861025
Remove verify from WALLET extension
lucksus Feb 20, 2024
9b9d6df
WIP signature extension
lucksus Feb 20, 2024
6ee9d6c
Fix signatures return/error types
lucksus Feb 20, 2024
e829b3a
create_signed_expression() and did functions in Rust
lucksus Feb 20, 2024
a72189d
agent JS extension
lucksus Feb 21, 2024
1cb7e2c
Replace all JS Signature code with Rust ext calls + use AGENT ext. in…
lucksus Feb 21, 2024
1ceb0da
Register new extensions
lucksus Feb 21, 2024
255ed31
Signature/verify roundtrip tests
lucksus Feb 21, 2024
e891d6b
Fix return values of SIGNATURE.verify
lucksus Feb 22, 2024
0c5c623
Merge branch 'dev' into rust-signatures
lucksus Feb 22, 2024
0b46f9e
Move tagExpressionSignatureStatus to utils
lucksus Feb 22, 2024
3fc461b
Ensure all link expressions have signature valid tags
lucksus Feb 22, 2024
98351ff
Adjust assertion to include sig tags
lucksus Feb 22, 2024
c8e7c2e
For removing links from DB, don’t expect LinkExpression to be exactly…
lucksus Feb 24, 2024
d3ac986
Fix breaking debug log when zome call payload can’t be converted to s…
lucksus Feb 24, 2024
4901fc8
Add tests that ensures signature verification can handle changed sort…
lucksus Feb 24, 2024
0a0358b
Make sure signature verify can handle different sortings
lucksus Feb 24, 2024
39c8adb
Add signature test for string data
lucksus Feb 26, 2024
8e2e525
Use Zulu format for timestamps
lucksus Feb 26, 2024
044c7b7
Decouple signature verification from timestamp format
lucksus Feb 26, 2024
20f54f2
Fix signature tags in perspective link subscriptions
lucksus Feb 26, 2024
485db3a
Add signature tags for links incoming from link language
lucksus Feb 26, 2024
a08ed46
Merge branch 'dev' into rust-signatures
lucksus Feb 26, 2024
3ce4281
Merge branch 'dev' into rust-signatures
lucksus Feb 26, 2024
898dd88
Merge branch 'dev' into rust-signatures
lucksus Feb 27, 2024
7de9c37
Remove definition of remove WALLET.verify() ext. function
lucksus Feb 27, 2024
922f33d
Make Expression<T> generic
lucksus Feb 27, 2024
248c96e
Merge branch 'dev' into rust-signatures
lucksus Feb 27, 2024
5413f5e
Deactivate signature sorting test since we now sort JsonValue in the …
lucksus Feb 27, 2024
a6001fe
Add missing file
lucksus Feb 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

56 changes: 56 additions & 0 deletions executor/src/agent_extension.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
export interface VerificationMethod {
id: string;
type: string;
controller: string;
publicKeyBase58?: string;
publicKeyMultibase?: string;
publicKeyJwk?: any; // Adjusted to 'any' since the specific structure is not provided
// Add other fields from the did_key::VerificationMethod if they exist
}

export interface DidDocument {
"@context": string | string[];
id: string;
alsoKnownAs?: string[];
controller?: string | string[];
verificationMethod: VerificationMethod[];
authentication?: (string | VerificationMethod)[];
assertionMethod?: (string | VerificationMethod)[];
keyAgreement?: (string | VerificationMethod)[];
capabilityInvocation?: (string | VerificationMethod)[];
capabilityDelegation?: (string | VerificationMethod)[];
service?: ServiceEndpoint[];
// Add other fields from the did_key::Document if they exist
}

export interface ServiceEndpoint {
id: string;
type: string;
serviceEndpoint: string;
// Add other fields from the did_key::ServiceEndpoint if they exist
}

export interface Expression {
author: string;
timestamp: string;
data: any;
proof: ExpressionProof;
}

export interface ExpressionProof {
signature: string;
key: string;
}

declare global {
interface RustAgent {
didDocument: () => DidDocument;
signingKeyId: () => string;
did: () => string;
createSignedExpression: (data: any) => Expression;
sign: (payload: Uint8Array) => Uint8Array;
signStringHex: (payload: string) => string;
}

const AGENT: RustAgent;
}
13 changes: 1 addition & 12 deletions executor/src/core/Ad4mCore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ import PerspectivesController from './PerspectivesController'
import LanguageController from './LanguageController'
import * as DIDs from './agent/DIDs'
import type { DIDResolver } from './agent/DIDs'
import Signatures from './agent/Signatures'
import * as PubSubDefinitions from './graphQL-interface/SubscriptionDefinitions'
import EntanglementProofController from './EntanglementProof'
import fs from 'node:fs'
Expand Down Expand Up @@ -58,7 +57,6 @@ export default class Ad4mCore {

#db: Ad4mDb
#didResolver: DIDResolver
#signatures: Signatures

#perspectivesController?: PerspectivesController
#languageController?: LanguageController
Expand All @@ -80,7 +78,6 @@ export default class Ad4mCore {
this.#agentService.load()
this.#db = Db.init(this.#config.dataPath)
this.#didResolver = DIDs.init(this.#config.dataPath)
this.#signatures = new Signatures()
const that = this
this.#resolveLanguagesReady = () => {}
this.#languagesReady = new Promise(resolve => {
Expand Down Expand Up @@ -134,13 +131,6 @@ export default class Ad4mCore {
return this.#runtimeService
}

get signatureService(): Signatures {
if (!this.#signatures) {
throw Error("No signature service")
}
return this.#signatures
}

get perspectivesController(): PerspectivesController {
if (!this.#perspectivesController) {
throw Error("No perspectiveController")
Expand Down Expand Up @@ -233,10 +223,9 @@ export default class Ad4mCore {
agent: this.#agentService,
runtime: this.#runtimeService,
//IPFS: this.#IPFS,
signatures: this.#signatures,
ad4mSignal: this.languageSignal,
config: this.#config,
}, { holochainService: this.#holochain!, runtimeService: this.#runtimeService, signatures: this.#signatures, db: this.#db } )
}, { holochainService: this.#holochain!, runtimeService: this.#runtimeService, db: this.#db } )

this.#perspectivesController = new PerspectivesController(this.#config.rootConfigPath, {
db: this.#db,
Expand Down
31 changes: 2 additions & 29 deletions executor/src/core/LanguageController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,9 @@ import * as PubSubDefinitions from './graphQL-interface/SubscriptionDefinitions'
import yaml from "js-yaml";
import { v4 as uuidv4 } from 'uuid';
import RuntimeService from './RuntimeService';
import Signatures from './agent/Signatures';
import { Ad4mDb } from './db';
import stringify from 'json-stable-stringify'
import { getPubSub } from './utils';
import { getPubSub, tagExpressionSignatureStatus } from './utils';

function cloneWithoutCircularReferences(obj: any, seen: WeakSet<any> = new WeakSet()): any {
if (typeof obj === 'object' && obj !== null) {
Expand Down Expand Up @@ -46,7 +45,6 @@ type SyncStateChangeObserver = (state: PerspectiveState, lang: LanguageRef)=>voi
interface Services {
holochainService: HolochainService,
runtimeService: RuntimeService,
signatures: Signatures,
db: Ad4mDb
}

Expand Down Expand Up @@ -87,7 +85,6 @@ export default class LanguageController {
#syncStateChangeObservers: SyncStateChangeObserver[];
#holochainService: HolochainService
#runtimeService: RuntimeService;
#signatures: Signatures;
#db: Ad4mDb;
#config: Config.MainConfig;
#pubSub: PubSub;
Expand All @@ -101,7 +98,6 @@ export default class LanguageController {
this.#context = context
this.#holochainService = services.holochainService
this.#runtimeService = services.runtimeService
this.#signatures = services.signatures
this.#db = services.db
this.#languages = new Map()
this.#languages.set("literal", {
Expand Down Expand Up @@ -1094,30 +1090,7 @@ export default class LanguageController {

async tagExpressionSignatureStatus(expression: Expression) {
if(expression) {
try{
await Signatures.tagExpressionSignatureStatus(expression)
} catch(e) {
let expressionFormatted;
if (typeof expression === "string") {
expressionFormatted = expression.substring(0, 50);
} else if (typeof expression === "object") {
let expressionString = JSON.stringify(expression);
expressionFormatted = expressionString.substring(0, 50)
} else {
expressionFormatted = expression;
}
let errMsg = `Error trying to verify signature for expression: ${expressionFormatted}`
console.error(errMsg)
console.error(e)
await this.#pubSub.publish(
PubSubDefinitions.EXCEPTION_OCCURRED_TOPIC,
{
title: "Failed to get expression",
message: errMsg,
type: ExceptionType.ExpressionIsNotVerified,
} as ExceptionInfo
);
}
tagExpressionSignatureStatus(expression)
}
}

Expand Down
41 changes: 28 additions & 13 deletions executor/src/core/Perspective.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,10 @@ import { MainConfig } from "./Config";
import { Mutex } from 'async-mutex'
import { DID } from "@coasys/ad4m/lib/src/DID";
import { Ad4mDb } from "./db";
import { getPubSub } from "./utils";
import Signatures from "./agent/Signatures";
import { getPubSub, tagExpressionSignatureStatus, removeSignatureTags } from "./utils";

const maxRetries = 10;
const backoffStep = 200;

export default class Perspective {
name?: string;
uuid?: string;
Expand Down Expand Up @@ -191,7 +189,7 @@ export default class Perspective {
}

linkToExpression(link: Link): Expression {
return this.#agent.createSignedExpression(link)
return AGENT.createSignedExpression(link)
}

ensureLinkExpression(maybeLink: any): Expression {
Expand All @@ -206,6 +204,12 @@ export default class Perspective {
throw new Error(`NH [${this.sharedUrl}] (${this.name}): Object is neither Link nor Expression: ${JSON.stringify(maybeLink)}`)
}

sanitizeLinkInput(link: (LinkInput | LinkExpressionInput)): LinkExpression {
removeSignatureTags(link)
return this.ensureLinkExpression(link)
}


private async getLinksAdapter(): Promise<LinkSyncAdapter | undefined> {
if(!this.neighbourhood || !this.neighbourhood.data.linkLanguage) {
//console.warn("Perspective.callLinksAdapter: Did not find neighbourhood or linkLanguage for neighbourhood on perspective, returning empty array")
Expand Down Expand Up @@ -421,7 +425,7 @@ export default class Perspective {
}

async addLink(link: LinkInput | LinkExpressionInput, status: LinkStatus = 'shared'): Promise<LinkExpression> {
const linkExpression = this.ensureLinkExpression(link);
const linkExpression = this.sanitizeLinkInput(link);

if (status === 'shared') {
const diff = {
Expand All @@ -441,19 +445,20 @@ export default class Perspective {
let perspectivePlain = this.plain();

linkExpression.status = status;
tagExpressionSignatureStatus(linkExpression);

await this.#pubSub.publish(PubSubDefinitions.LINK_ADDED_TOPIC, {
perspective: perspectivePlain,
link: linkExpression
})
this.#prologNeedsRebuild = true



return linkExpression
}

async addLinks(links: (LinkInput | LinkExpressionInput)[], status: LinkStatus = 'shared'): Promise<LinkExpression[]> {
const linkExpressions = links.map(l => this.ensureLinkExpression(l));
let linkExpressions = links.map(l => this.sanitizeLinkInput(l))

const diff = {
additions: linkExpressions,
removals: []
Expand All @@ -466,6 +471,12 @@ export default class Perspective {

await this.#db.addManyLinks(this.uuid!, linkExpressions, status);
this.#prologNeedsRebuild = true;

linkExpressions = linkExpressions.map(l => {
tagExpressionSignatureStatus(l);
return {...l, status} as LinkExpression
})

let perspectivePlain = this.plain();
for (const link of linkExpressions) {
await this.#pubSub.publish(PubSubDefinitions.LINK_ADDED_TOPIC, {
Expand All @@ -476,11 +487,11 @@ export default class Perspective {
this.#prologNeedsRebuild = true

// @ts-ignore
return linkExpressions.map(l => ({...l, status}))
return linkExpressions
}

async removeLinks(links: LinkInput[]): Promise<LinkExpression[]> {
const linkExpressions = links.map(l => this.ensureLinkExpression(l));
const linkExpressions = links.map(l => this.sanitizeLinkInput(l));
const diff = {
additions: [],
removals: linkExpressions
Expand Down Expand Up @@ -536,10 +547,10 @@ export default class Perspective {
}

async updateLink(oldLink: LinkExpressionInput, newLink: LinkInput) {
this.sanitizeLinkInput(oldLink);
const link = await this.#db.getLink(this.uuid!, oldLink);
if (!link) {
const allLinks = await this.#db.getAllLinks(this.uuid!);
throw new Error(`NH [${this.sharedUrl}] (${this.name}) Link not found in perspective "${this.plain()}": ${JSON.stringify(oldLink)}`)
throw new Error(`NH [${this.sharedUrl}] (${this.name}) Link not found in perspective "${this.plain().uuid}": ${JSON.stringify(oldLink)}`)
}

const newLinkExpression = this.ensureLinkExpression(newLink)
Expand All @@ -556,6 +567,8 @@ export default class Perspective {
await this.#db.addPendingDiff(this.uuid!, diff);
}

tagExpressionSignatureStatus(newLinkExpression);

const perspective = this.plain();
this.#prologNeedsRebuild = true;
await this.#pubSub.publish(PubSubDefinitions.LINK_UPDATED_TOPIC, {
Expand All @@ -568,6 +581,8 @@ export default class Perspective {
newLinkExpression.status = link.status
}



return newLinkExpression
}

Expand Down Expand Up @@ -694,7 +709,7 @@ export default class Perspective {

for(const link of values) {
try {
await Signatures.tagExpressionSignatureStatus(link)
await tagExpressionSignatureStatus(link)
} catch (e) {
console.error(`Perspective.getLinks(): NH [${this.sharedUrl}] (${this.name}): Got error when trying to tag expression signature status: ${e}`);
link.proof.invalid = true;
Expand Down
4 changes: 3 additions & 1 deletion executor/src/core/PerspectivesController.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import fs from 'node:fs'
import { v4 as uuidv4 } from 'uuid';
import * as PubSubDefinitions from './graphQL-interface/SubscriptionDefinitions'
import type PerspectiveContext from './PerspectiveContext'
import { Perspective as Ad4mPerspective, Neighbourhood, LinkQuery, PerspectiveHandle, LanguageRef, PerspectiveDiff, PerspectiveState, PerspectiveExpression, NeighbourhoodExpression } from '@coasys/ad4m'
import { Perspective as Ad4mPerspective, Neighbourhood, LinkQuery, PerspectiveHandle, LanguageRef, PerspectiveDiff, PerspectiveState, PerspectiveExpression, NeighbourhoodExpression, LinkExpression } from '@coasys/ad4m'
import Perspective from './Perspective'
import { getPubSub, sleep } from './utils';

Expand Down Expand Up @@ -47,13 +47,15 @@ export default class PerspectivesController {
try {
let perspectivePlain = perspective.plain();
for (const link of diff.additions) {
tagExpressionSignatureStatus(link);
await this.#pubSub.publish(PubSubDefinitions.LINK_ADDED_TOPIC, {
perspective: perspectivePlain,
link: link
})
}

for (const linkRemoved of diff.removals) {
tagExpressionSignatureStatus(linkRemoved);
await this.#pubSub.publish(PubSubDefinitions.LINK_REMOVED_TOPIC, {
perspective: perspectivePlain,
link: linkRemoved
Expand Down
Loading