Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/1897 psk #1961

Merged
merged 39 commits into from
Nov 14, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7199e08
Generate psk when creating community; add psk connection protector to…
EmiM Oct 6, 2023
1402f58
Merge branch 'develop' into feature/1897-psk
EmiM Oct 10, 2023
bb49437
Add psk to invitation link
EmiM Oct 12, 2023
e1abeb9
Retrieve psk from invitation link
EmiM Oct 16, 2023
b1d41bd
Merge branch 'develop' into feature/1897-psk
EmiM Oct 18, 2023
30baa07
Fix import
EmiM Oct 18, 2023
ba678ee
Refactor invitation url parsing functions
EmiM Oct 18, 2023
f4c7370
Fix import
EmiM Oct 24, 2023
4a843cf
Merge branch 'develop' into feature/1897-psk
EmiM Oct 24, 2023
294945c
feat: Check if psk in invitation link is base64 encoded #1897
EmiM Oct 24, 2023
33d4b6b
feat: check psk length in invitation code
EmiM Oct 24, 2023
800b4f3
fix: generate valid psk for backend test libp2p param
EmiM Oct 24, 2023
9f3680a
fix: adjust tests for desktop and e2e
EmiM Oct 24, 2023
ac0843d
fix: handle parsing invitation code errors in main.ts
EmiM Oct 24, 2023
27fb780
fix: mobile tests; handle invalid deep link
EmiM Oct 24, 2023
c00c8f9
fix: desktop rtl tests
EmiM Oct 24, 2023
ffdd901
fix: mobile deep link saga - check if invitation code is parseable be…
EmiM Oct 25, 2023
010da05
feat: move valid invitation url test data to common package; fix: mob…
EmiM Oct 25, 2023
00fbf5d
fix: backend unit tests with tor
EmiM Oct 25, 2023
ae890e2
fix: e2e test for invitation link
EmiM Oct 25, 2023
e8586e2
fix: desktop test
EmiM Oct 25, 2023
46eb6fd
test: temporarily skip backward compatibility e2e test - psk changes …
EmiM Oct 26, 2023
857bd78
Merge branch 'develop' into feature/1897-psk
EmiM Oct 26, 2023
967210e
chore: update detox version - fixes 'tap' https://github.com/wix/Deto…
EmiM Oct 26, 2023
ef603aa
chore: move creating libp2p psk to libp2p.service
EmiM Oct 30, 2023
4e5cd23
fix: generating libp2p psk after refactoring
EmiM Oct 30, 2023
9a7fc45
chore: cleanup, renaming
EmiM Oct 31, 2023
3e1f0f3
chore: move statics related to invitation code to invitationCode file
EmiM Oct 31, 2023
68b4139
Merge branch 'develop' into feature/1897-psk
EmiM Oct 31, 2023
96921cb
chore: link app name with displayed log in e2e tests
EmiM Oct 31, 2023
f99caf3
chore: increase timeout for multiple clients e2e test
EmiM Nov 2, 2023
442319e
chore: cleanup logs
EmiM Nov 2, 2023
3533a16
chore: update changelog
EmiM Nov 2, 2023
a671a63
test: add log to check android e2e test failing
EmiM Nov 2, 2023
eda1888
chore: remove logs from mobile e2e test
EmiM Nov 7, 2023
a15342d
Merge branch 'develop' into feature/1897-psk
EmiM Nov 7, 2023
d726933
Merge branch 'develop' into feature/1897-psk
EmiM Nov 10, 2023
0ec7928
Merge branch 'develop' into feature/1897-psk
EmiM Nov 14, 2023
94b0978
chore: update package-lock
EmiM Nov 14, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74,162 changes: 37,084 additions & 37,078 deletions packages/backend/package-lock.json

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions packages/backend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
"@types/orbit-db": "git+https://github.com/orbitdb/orbit-db-types.git",
"@types/supertest": "^2.0.11",
"@types/tmp": "^0.2.3",
"@types/validator": "^13.1.4",
"@types/validator": "^13.11.5",
"@types/ws": "8.5.3",
"babel-jest": "^29.3.1",
"cross-env": "^5.2.0",
Expand Down Expand Up @@ -134,7 +134,7 @@
"socks-proxy-agent": "^5.0.0",
"string-replace-loader": "3.1.0",
"ts-jest-resolver": "^2.0.0",
"validator": "^13.6.0"
"validator": "^13.11.0"
leblowl marked this conversation as resolved.
Show resolved Hide resolved
},
"overrides": {
"level": "$level",
Expand Down
7 changes: 4 additions & 3 deletions packages/backend/src/nest/common/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ import crypto from 'crypto'
import { type PermsData } from '@quiet/types'
import { TestConfig } from '../const'
import logger from './logger'
import { createCertificatesTestHelper } from './client-server'
import { Libp2pNodeParams } from '../libp2p/libp2p.types'
import { createLibp2pAddress, createLibp2pListenAddress } from '@quiet/common'
import { Libp2pService } from '../libp2p/libp2p.service'

const log = logger('test')

export interface Ports {
Expand Down Expand Up @@ -189,20 +190,20 @@ export const testBootstrapMultiaddrs = [
]

export const libp2pInstanceParams = async (): Promise<Libp2pNodeParams> => {
const pems = await createCertificatesTestHelper('address1.onion', 'address2.onion')
const port = await getPort()
const peerId = await createPeerId()
const address = '0.0.0.0'
const peerIdRemote = await createPeerId()
const remoteAddress = createLibp2pAddress(address, peerIdRemote.toString())

const libp2pKey = Libp2pService.generateLibp2pPSK().fullKey
return {
peerId,
listenAddresses: [createLibp2pListenAddress('localhost')],
agent: createHttpsProxyAgent({ port: 1234, host: 'localhost' }),
localAddress: createLibp2pAddress('localhost', peerId.toString()),
targetPort: port,
peers: [remoteAddress],
psk: libp2pKey,
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -87,8 +87,6 @@ beforeEach(async () => {
localDbService = await module.resolve(LocalDbService)
registrationService = await module.resolve(RegistrationService)
tor = await module.resolve(Tor)

console.log('tor ', tor)
await tor.init()

const torPassword = crypto.randomBytes(16).toString('hex')
Expand All @@ -106,6 +104,9 @@ beforeEach(async () => {
connectionsManagerService.libp2pService = libp2pService

quietDir = await module.resolve(QUIET_DIR)

const pskBase64 = Libp2pService.generateLibp2pPSK().psk
await localDbService.put(LocalDBKeys.PSK, pskBase64)
})

afterEach(async () => {
Expand All @@ -117,11 +118,6 @@ afterEach(async () => {
})

describe('Connections manager', () => {
it('runs tor by default', async () => {
await connectionsManagerService.init()
console.log(connectionsManagerService.isTorInit)
})

it('saves peer stats when peer has been disconnected', async () => {
class RemotePeerEventDetail {
peerId: string
Expand All @@ -135,7 +131,6 @@ describe('Connections manager', () => {
}
}
const emitSpy = jest.spyOn(libp2pService, 'emit')
// const emitSpy = jest.spyOn(libp2pService, 'emit')

const launchCommunityPayload: InitCommunityPayload = {
id: community.id,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { EventEmitter } from 'events'
import getPort from 'get-port'
import PeerId from 'peer-id'
import { removeFilesFromDir } from '../common/utils'
import validator from 'validator'
EmiM marked this conversation as resolved.
Show resolved Hide resolved
import {
AskForMessagesPayload,
ChannelMessagesIdsResponse,
Expand Down Expand Up @@ -59,6 +60,9 @@ import { StorageEvents } from '../storage/storage.types'
import { LazyModuleLoader } from '@nestjs/core'
import Logger from '../common/logger'
import { emitError } from '../socket/socket.errors'
import { toString as uint8ArrayToString } from 'uint8arrays/to-string'
import { fromString as uint8ArrayFromString } from 'uint8arrays/from-string'
EmiM marked this conversation as resolved.
Show resolved Hide resolved
import { isPSKcodeValid } from '@quiet/common'

@Injectable()
export class ConnectionsManagerService extends EventEmitter implements OnModuleInit {
Expand Down Expand Up @@ -274,11 +278,35 @@ export class ConnectionsManagerService extends EventEmitter implements OnModuleI
},
network,
}
const psk = community.psk
if (psk) {
console.log('createNetwork got psk', psk)
if (!isPSKcodeValid(psk)) {
emitError(this.serverIoProvider.io, {
type: SocketActionTypes.NETWORK,
message: ErrorMessages.NETWORK_SETUP_FAILED,
community: community.id,
})
return
}
await this.localDbService.put(LocalDBKeys.PSK, psk)
}

this.serverIoProvider.io.emit(SocketActionTypes.NETWORK, payload)
}

private async generatePSK() {
const pskBase64 = Libp2pService.generateLibp2pPSK().psk
await this.localDbService.put(LocalDBKeys.PSK, pskBase64)
console.log('psk base64 SAVED', pskBase64)
this.serverIoProvider.io.emit(SocketActionTypes.PSK, { psk: pskBase64 })
}

public async createCommunity(payload: InitCommunityPayload) {
console.log('ConnectionsManager.createCommunity peers:', payload.peers)

await this.generatePSK()

await this.launchCommunity(payload)
this.logger(`Created and launched community ${payload.id}`)
this.serverIoProvider.io.emit(SocketActionTypes.NEW_COMMUNITY, { id: payload.id })
Expand Down Expand Up @@ -345,6 +373,13 @@ export class ConnectionsManagerService extends EventEmitter implements OnModuleI
if (!peers || peers.length === 0) {
peers = [this.libp2pService.createLibp2pAddress(onionAddress, _peerId.toString())]
}
const pskValue: string = await this.localDbService.get(LocalDBKeys.PSK)
if (!pskValue) {
throw new Error('No psk in local db')
}
console.log('psk base64 RETRIEVED', pskValue)

const libp2pPSK = Libp2pService.generateLibp2pPSK(pskValue).fullKey

const params: Libp2pNodeParams = {
peerId: _peerId,
Expand All @@ -353,16 +388,15 @@ export class ConnectionsManagerService extends EventEmitter implements OnModuleI
localAddress: this.libp2pService.createLibp2pAddress(onionAddress, _peerId.toString()),
targetPort: this.ports.libp2pHiddenService,
peers,
psk: libp2pPSK,
}

await this.libp2pService.createInstance(params)
// KACPER
// Libp2p event listeners
this.libp2pService.on(Libp2pEvents.PEER_CONNECTED, (payload: { peers: string[] }) => {
this.serverIoProvider.io.emit(SocketActionTypes.PEER_CONNECTED, payload)
})
this.libp2pService.on(Libp2pEvents.PEER_DISCONNECTED, async (payload: NetworkDataPayload) => {
console.log(' this.libp2pService.on(Libp2pEvents.PEER_DISCONNECTED')
const peerPrevStats = await this.localDbService.find(LocalDBKeys.PEERS, payload.peer)
const prev = peerPrevStats?.connectionTime || 0

Expand Down Expand Up @@ -420,6 +454,7 @@ export class ConnectionsManagerService extends EventEmitter implements OnModuleI
}
})
this.socketService.on(SocketActionTypes.CREATE_NETWORK, async (args: Community) => {
console.log('CREATE NETWORK', args)
await this.createNetwork(args)
})
this.socketService.on(SocketActionTypes.CREATE_COMMUNITY, async (args: InitCommunityPayload) => {
Expand Down
31 changes: 31 additions & 0 deletions packages/backend/src/nest/libp2p/libp2p.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ import Logger from '../common/logger'
import { webSockets } from '../websocketOverTor'
import { all } from '../websocketOverTor/filters'
import { createLibp2pAddress, createLibp2pListenAddress } from '@quiet/common'
import { preSharedKey } from 'libp2p/pnet'
import { toString as uint8ArrayToString } from 'uint8arrays/to-string'
import { fromString as uint8ArrayFromString } from 'uint8arrays/from-string'
import crypto from 'crypto'

const KEY_LENGTH = 32

@Injectable()
export class Libp2pService extends EventEmitter {
Expand All @@ -44,6 +50,28 @@ export class Libp2pService extends EventEmitter {
return createLibp2pListenAddress(address)
}

public static generateLibp2pPSK(key?: string) {
/**
* Based on 'libp2p/pnet' generateKey
*
* @param key: base64 encoded psk
*/
const libp2pPSK = new Uint8Array(95)
let psk
if (key) {
psk = uint8ArrayFromString(key, 'base64')
} else {
psk = crypto.randomBytes(KEY_LENGTH)
}

const base16StringKey = uint8ArrayToString(psk, 'base16')
const fullKey = uint8ArrayFromString('/key/swarm/psk/1.0.0/\n/base16/\n' + base16StringKey)

libp2pPSK.set(fullKey)
EmiM marked this conversation as resolved.
Show resolved Hide resolved

return { psk: psk.toString('base64'), fullKey }
}

public async createInstance(params: Libp2pNodeParams): Promise<Libp2p> {
if (this.libp2pInstance) {
return this.libp2pInstance
Expand All @@ -64,6 +92,9 @@ export class Libp2pService extends EventEmitter {
addresses: {
listen: params.listenAddresses,
},
connectionProtector: preSharedKey({
psk: params.psk,
}),
streamMuxers: [mplex()],
connectionEncryption: [noise()],
relay: {
Expand Down
1 change: 1 addition & 0 deletions packages/backend/src/nest/libp2p/libp2p.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ export interface Libp2pNodeParams {
localAddress: string
targetPort: number
peers: string[]
psk: Uint8Array
}

export interface InitLibp2pParams {
Expand Down
1 change: 1 addition & 0 deletions packages/backend/src/nest/local-db/local-db.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ export enum LocalDBKeys {
COMMUNITY = 'community',
REGISTRAR = 'registrar',
PEERS = 'peers',
PSK = 'psk',
EmiM marked this conversation as resolved.
Show resolved Hide resolved
}
export type LocalDbStatus = 'opening' | 'open' | 'closing' | 'closed'
4 changes: 4 additions & 0 deletions packages/backend/src/nest/socket/socket.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,10 @@ export class SocketService extends EventEmitter implements OnModuleInit {
this.logger('Leaving community')
this.emit(SocketActionTypes.LEAVE_COMMUNITY)
})
socket.on(SocketActionTypes.PSK, payload => {
this.logger('Saving PSK', payload)
this.emit(SocketActionTypes.PSK, payload)
})
})
}

Expand Down
29 changes: 28 additions & 1 deletion packages/common/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion packages/common/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,16 @@
"@quiet/eslint-config": "^2.0.2-alpha.0",
"@types/jest": "^26.0.23",
"@types/node": "^17.0.21",
"@types/validator": "^13.11.5",
"jest": "^26.6.3",
"ts-jest": "^26.5.2",
"typescript": "^4.9.3"
},
"dependencies": {
"@quiet/types": "^2.0.2-alpha.0",
"cross-env": "^5.2.0",
"debug": "^4.3.1"
"debug": "^4.3.1",
"validator": "^13.11.0"
},
"jest": {
"transform": {
Expand Down
1 change: 1 addition & 0 deletions packages/common/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ export * from './channelAddress'
export * from './naming'
export * from './fileData'
export * from './libp2p'
export * from './tests'
Loading
Loading