Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: remote engine management #4364

Merged
merged 31 commits into from
Jan 14, 2025
Merged
Changes from 1 commit
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
b7a86a0
feat: remote engine management
urmauur Dec 30, 2024
68e05ee
chore: fix linter issue
urmauur Dec 30, 2024
323700f
chore: remove unused imports
louis-menlo Jan 2, 2025
22e9db0
fix: populate engines, models and legacy settings (#4403)
louis-menlo Jan 6, 2025
b62db41
fix: check exist path before reading
louis-menlo Jan 6, 2025
3eb4295
fix: engines and models persist - race condition
louis-menlo Jan 6, 2025
3e48008
chore: update issue state
urmauur Jan 6, 2025
dc84557
test: update test cases
louis-menlo Jan 6, 2025
7fd8950
chore: bring back Cortex extension settings
louis-menlo Jan 6, 2025
e108f49
chore: setup button gear / plus based apikey
urmauur Jan 6, 2025
3fa8f41
chore: fix remote engine from welcome screen
urmauur Jan 8, 2025
7fd0dd6
chore: resolve linter issue
urmauur Jan 8, 2025
0df0983
chore: support request headers template
louis-menlo Jan 8, 2025
db75c9b
chore: update engines using header_template instead of api_key_template
louis-menlo Jan 8, 2025
a50582a
chore: update models on changes
louis-menlo Jan 8, 2025
5d8d4b6
fix: anthropic response template
louis-menlo Jan 8, 2025
f47392b
chore: fix welcome screen and debounce update value input
urmauur Jan 8, 2025
0468afb
chore: update engines list on changes
louis-menlo Jan 8, 2025
8e6ed11
chore: update engines list on change
louis-menlo Jan 8, 2025
67625df
chore: update desc form add modal remote engines
urmauur Jan 8, 2025
c11ec3c
chore: bump cortex version to latest RC
louis-menlo Jan 13, 2025
06188a2
chore: fix linter
louis-menlo Jan 13, 2025
a15d2ba
fix: transform payload of Anthropic and OpenAI
louis-menlo Jan 13, 2025
9094d02
fix: typo
urmauur Jan 14, 2025
5d56d1e
fix: openrouter model id for auto routing
louis-menlo Jan 14, 2025
dffb37d
chore: remove remote engine URL setting
louis-menlo Jan 14, 2025
fe67931
chore: add cohere engine and model support
louis-menlo Jan 14, 2025
a54e0f6
fix: should not clean on app launch - models list display issue
louis-menlo Jan 14, 2025
3511583
fix: local engine check logic
louis-menlo Jan 14, 2025
ca55df3
chore: bump app version to latest release 0.5.13
louis-menlo Jan 14, 2025
a63322b
test: fix failed tests
louis-menlo Jan 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
test: fix failed tests
louis-menlo committed Jan 14, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
commit a63322b189b93f4d2a7e72b1973db3e9556a2156
45 changes: 17 additions & 28 deletions web/utils/modelEngine.test.ts
Original file line number Diff line number Diff line change
@@ -25,40 +25,29 @@ describe('isLocalEngine', () => {

it('should return false if engine is not found', () => {
mockGet.mockReturnValue(null)
const result = isLocalEngine('nonexistentEngine')
const result = isLocalEngine(
{
'llama-cpp': [],
} as any,
'nonexistentEngine'
)
expect(result).toBe(false)
})

it('should return true if engine is an instance of LocalOAIEngine', () => {
const mockEngineObj = {
__proto__: {
constructor: {
__proto__: {
name: LocalOAIEngine.name,
it('should return true if this is a local engine', () => {
mockGet.mockReturnValue(null)
const result = isLocalEngine(
{
'llama-cpp': [
{
type: 'local',
},
},
},
}
mockGet.mockReturnValue(mockEngineObj)
const result = isLocalEngine('localEngine')
],
} as any,
'llama-cpp'
)
expect(result).toBe(true)
})

it('should return false if engine is not an instance of LocalOAIEngine', () => {
const mockEngineObj = {
__proto__: {
constructor: {
__proto__: {
name: 'SomeOtherEngine',
},
},
},
}
mockGet.mockReturnValue(mockEngineObj)
const result = isLocalEngine('someOtherEngine')
expect(result).toBe(false)
})

jest.mock('@janhq/core', () => ({
...jest.requireActual('@janhq/core'),
EngineManager: {

Unchanged files with check annotations Beta

useEffect(() => {
configurePullOptions()
}, [])

Check warning on line 36 in web/hooks/useConfigurations.ts

GitHub Actions / test-on-macos

React Hook useEffect has a missing dependency: 'configurePullOptions'. Either include it or remove the dependency array

Check warning on line 36 in web/hooks/useConfigurations.ts

GitHub Actions / test-on-ubuntu

React Hook useEffect has a missing dependency: 'configurePullOptions'. Either include it or remove the dependency array

Check warning on line 36 in web/hooks/useConfigurations.ts

GitHub Actions / test-on-windows-pr

React Hook useEffect has a missing dependency: 'configurePullOptions'. Either include it or remove the dependency array

Check warning on line 36 in web/hooks/useConfigurations.ts

GitHub Actions / coverage-check

React Hook useEffect has a missing dependency: 'configurePullOptions'. Either include it or remove the dependency array
return {
configurePullOptions,
engines &&
Object.entries(engines)
.filter(([key]) => !isLocalEngine(engines, key as InferenceEngine))
.flatMap(([_, engineArray]) => engineArray as EngineConfig)

Check warning on line 22 in web/hooks/useStarterScreen.ts

GitHub Actions / test-on-macos

'_' is defined but never used

Check warning on line 22 in web/hooks/useStarterScreen.ts

GitHub Actions / test-on-ubuntu

'_' is defined but never used

Check warning on line 22 in web/hooks/useStarterScreen.ts

GitHub Actions / test-on-windows-pr

'_' is defined but never used

Check warning on line 22 in web/hooks/useStarterScreen.ts

GitHub Actions / coverage-check

'_' is defined but never used
const isDownloadALocalModel = useMemo(
() =>
engines &&
Object.entries(engines)
.filter(([key]) => key === name)
.flatMap(([_, engineArray]) => engineArray as EngineConfig)[0]

Check warning on line 42 in web/screens/Settings/Engines/RemoteEngineSettings.tsx

GitHub Actions / test-on-macos

'_' is defined but never used

Check warning on line 42 in web/screens/Settings/Engines/RemoteEngineSettings.tsx

GitHub Actions / test-on-ubuntu

'_' is defined but never used

Check warning on line 42 in web/screens/Settings/Engines/RemoteEngineSettings.tsx

GitHub Actions / test-on-windows-pr

'_' is defined but never used

Check warning on line 42 in web/screens/Settings/Engines/RemoteEngineSettings.tsx

GitHub Actions / coverage-check

'_' is defined but never used
const debounceRef = useRef<NodeJS.Timeout | null>(null)
],
})
},
[

Check warning on line 84 in web/screens/Thread/ThreadCenterPanel/AssistantSetting/index.tsx

GitHub Actions / test-on-macos

React Hook useCallback has a missing dependency: 'resetGenerating'. Either include it or remove the dependency array

Check warning on line 84 in web/screens/Thread/ThreadCenterPanel/AssistantSetting/index.tsx

GitHub Actions / test-on-ubuntu

React Hook useCallback has a missing dependency: 'resetGenerating'. Either include it or remove the dependency array

Check warning on line 84 in web/screens/Thread/ThreadCenterPanel/AssistantSetting/index.tsx

GitHub Actions / test-on-windows-pr

React Hook useCallback has a missing dependency: 'resetGenerating'. Either include it or remove the dependency array

Check warning on line 84 in web/screens/Thread/ThreadCenterPanel/AssistantSetting/index.tsx

GitHub Actions / coverage-check

React Hook useCallback has a missing dependency: 'resetGenerating'. Either include it or remove the dependency array
activeAssistant,
activeThread,
componentData,
*/
export abstract class EngineManagementExtension extends BaseExtension {
type(): ExtensionTypeEnum | undefined {
return ExtensionTypeEnum.Engine

Check warning on line 18 in core/src/browser/extensions/enginesManagement.ts

GitHub Actions / coverage-check

18 line is not covered with tests
}
/**
let errorCode = ErrorCode.Unknown
if (data.error) {
errorCode = data.error.code ?? data.error.type ?? ErrorCode.Unknown
} else if (response.status === 401) {
errorCode = ErrorCode.InvalidApiKey

Check warning on line 40 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

39-40 lines are not covered with tests
}
const error = {
message: data.error?.message ?? data.message ?? 'Error occurred.',
) {
const data = await response.json()
if (data.error || data.message) {
subscriber.error(data.error ?? data)
subscriber.complete()
return

Check warning on line 60 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

58-60 lines are not covered with tests
}
if (transformResponse) {
subscriber.next(transformResponse(data))
for (const line of lines) {
try {
if (transformResponse) {
content += transformResponse(line)
subscriber.next(content ?? '')

Check warning on line 89 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

88-89 lines are not covered with tests
} else {
const toParse = cachedLines + line
if (!line.includes('data: [DONE]')) {
const data = JSON.parse(toParse.replace('data: ', ''))
if ('error' in data) {
subscriber.error(data.error)
subscriber.complete()
return

Check warning on line 97 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

95-97 lines are not covered with tests
}
content += data.choices[0]?.delta?.content ?? ''
if (content.startsWith('assistant: ')) {
content = content.replace('assistant: ', '')

Check warning on line 101 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

101 line is not covered with tests
}
if (content !== '') subscriber.next(content)
}
}
} catch {
cachedLines = line

Check warning on line 107 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

107 line is not covered with tests
}
}
}
}
subscriber.complete()
})
.catch((err) => subscriber.error(err))

Check warning on line 114 in core/src/browser/extensions/engines/helpers/sse.ts

GitHub Actions / coverage-check

114 line is not covered with tests
})
}
if (!fs.existsSync(configurationFile)) {
// create default app config if we don't have one
console.debug(

Check warning on line 21 in core/src/node/helper/config.ts

GitHub Actions / coverage-check

21 line is not covered with tests
`App config not found, creating default config at ${configurationFile}`
)
fs.writeFileSync(configurationFile, JSON.stringify(appDefaultConfiguration))
return appDefaultConfiguration

Check warning on line 25 in core/src/node/helper/config.ts

GitHub Actions / coverage-check

24-25 lines are not covered with tests
}
try {