- Fix S3 acceess by dropping aiobotocore patching and using public API instead. (#3155)
No significant changes.
-
Added cluster-bound apps configuration into the SDK. (#3142)
-
Prefer
.apolo
configuration folder and.apoloignore
files and folders over old.neuro
based names; old names are still supported. (#3145) -
Balance is no longer stored on a cluster level, and was moved to an organization level, e.g., to an org itself, and to an org users, instead of a cluster / cluster users.
New commands:
apolo admin set-org-defaults
- allows to set an organization default values, such as a default user credits
Existing commands changes:
apolo admin add-cluster-user
cmd is no longer accepting acredits
argument.apolo admin set-user-credits
cmd is now expecting an org name instead of a cluster name.apolo admin add-user-credits
cmd is now expecting an org name instead of a cluster name.apolo admin set-org-cluster-credits
was removed in a favor of anapolo admin set-org-credits
.apolo admin add-org-cluster-credits
was removed in a favor of anapolo admin add-org-credits
. (#3137)
- Stop using deprecated
ssl.PROTOCOL_TLS
in favor ofssl.PROTOCOL_TLS_CLIENT
(#3146)
No significant changes.
No significant changes.
No significant changes.
No significant changes.
No significant changes.
No significant changes.
No significant changes.
No significant changes.
- Support AMD, Intel GPUs (#3007)
No significant changes.
- Do not perform terminal settings reset in PyCharm while it causes screen cleanup. (#3005)
No significant changes.
No significant changes.
- Commands
neuro run
,neuro logs
,neuro attach
andneuro exec
in non-quiet mode now prints details for cancelled and failed jobs. Also improved other indications of the job status. (#2800) - Commands
neuro run
andneuro attach
now re-attach to restarted jobs. Previously they waited until the job finished. (#2802) - Added support of cluster energy schedules.
neuro config show --energy
will display awailable energy schedule periods.neuro run --schedule-name <energy-schedule-name>
will run the job within the specified . Note - the selected preset should have an enabled scheduler.neuro status <job-id>
will include the energy schedule name if one was used for running the job. (#2903) - Added Python 3.11 support. (#2908)
- Add
neuro config switch-project
command. (#2940) - Support projects in storage commands, make project directory default instead of user home directory. (#2949)
- Support projects in image commands. (#2950)
- Support projects in
neuro secret
cli. (#2952) - Support projects in
neuro blob
cli commands andneuro-sdk
. (#2954) - Add project support in
neuro job
commands. (#2955) - Support projects in disk CLI commands and SDK methods. (#2960)
- Support projects in service account CLI commands and SDK methods. (#2965)
- Unhide remove-project command. (#2976)
- Align all
ls
command options. (#2995)
- Fix human-readable datetime formatter to correctly report jobs lifespan ending in future (#2933)
- Update projects after fetching config from server. (#2992)
- Add update-node-pool command. Only idle_size update is available. (#2740)
- Integrate neuro-config-client package. (#2791)
- Added project management commands to
neuro admin
. (#2798)
- Fix
neuro admin update-resource-preset
: previous values are now used for boolean parameters if options--scheduler/--no-scheduler
or--preemptible-node/--no-preemptible-node
are not explicitly specified. (#2761)
- Update size formatting: use decimal prefixes (#2768)
- Fix formatting URIs in short form for users with organization. (#2749)
- Disks, buckets and secrets are now created with the current organization instead of no organization if
--org
is not explicitly specified. (#2756)
- Pin
neuro-admin-client
to>=22.6.4
, make SDK future-compatible with nextneuro-admin-client
releases. (#2757)
- Allow user without clusters call neuro-sdk Jobs.status method if they have permissions. (#2733)
- Re-load clusters config in
neuro config show
. (#2728)
-
Support organization storage in disk usage command when user switches to organization. (#2585)
-
Added
neuro admin set-org-cluster-defaults
command to allow org cluster manager to modify org cluster defaults. (#2602) -
Support custom storage path in disk usage command. (#2622)
-
neuro blob statbucket
now displays some additional information for imported buckets.- For AWS: "External name", "External endpoint", "External region name".
- For Azure: "External name", "External endpoint".
- For GCP: "External name". (#2706)
- Fix resolving job URIs containing organization name and/or service account name. (#2572)
- Provide click 8.1.3 support with keeping 8.0+ compatibility. (#2689)
- Support organization storage size parameter. (#2560)
- Added client-side validation of docker image name and tag. (#2525)
- Added support of column type
org_name
to ps-format. (#2533) - Added "ORG" column to get-cluster-users output. (#2541)
- Added info about org quota to
neuro config show
. (#2545)
- Cluster/org name validness now checked after config re-fetch in
neuro config switch-cluster
andneuro config switch-org
commands. (#2543)
-
Add command
neuro acl list-roles
to list roles created by user. (#2496) -
Added support to set/update cluster level default credits and quota.
Use options
--default-credits
and--default-jobs
of commandsneuro admin add-cluster
,neuro admin update-cluster
,neuro admin add-org-cluster
andneuro admin update-org-cluster
to set and update cluster defaults. This values will when new user with role "user" is added to cluster (either byneuro admin add-cluster-user
or if user registers himself using web interface). The default for managers and admins is unlimited quota and credits as the can edit their quota.You can override default value by using
--credits
and--jobs
options ofneuro admin add-org-cluster
command. (#2520)
- Fixed memory leak in
neuro blob cp
. (#2523)
- Added support
--owner
argument in blob bucket level commands to allow referring to another users bucket by name. (#2494) - Add --force/-f flag to neuro blob rmbucket to force remove non-empty bucket. (#2495)
- Support
neuro -q ls
andneuro -q blob ls
for quiet output enforcing. (#2506)
- Replace
--http
option with--http-port
, keep--http
option as a hidden deprecated alternative, scheduled for removal later. (#2501) - Remove deprecated
neuro project init
command. (#2502) - Remove
client.job.tags()
method from SDK andneuro job tags
command from CLI. (#2503) - Drop deprecated API from SDK (#2505)
- Sort CLI commands, groups and topics alphabetically. (#2488)
- Add support for Open-Sourced user-less services deployment. (#2492)
- Replace "Spend credits" with "Credits spent" in formatter. (#2435)
- Fix importing of asynccontextmanager from wrong place
-
Merge
config show-quota
intoconfig show
CLI command. (#2436) -
Added admin commands to manager orgs and org-clusters:
Managing orgs:
neuro admin get-orgs
Print the list of available orgs user has access to.neuro admin add-org <org_name>
Create a new org.neuro admin remove-org <org_name>
Drop a org. Removes all memberships, very dangerous operation.
Managing org members:
neuro admin get-org-users <org_name>
List all members of orgs.neuro admin add-org-user <org_name> <username>
Add user to the org.neuro admin remove-org-user <org_name> <username>
Remove user from the org.
Managing access of orgs to clusters:
neuro admin get-org-clusters <cluster_name>
Print the list of all orgs in the clusterneuro admin add-org-cluster <cluster_name> <org_name>
Add org access to specified clusterneuro admin get-org-cluster-quota <cluster_name> <org_name>
Get info about org quota in given clusterneuro admin set-org-cluster-quota [options] <cluster_name> <org_name>
Set org cluster quota to given valuesneuro admin set-org-cluster-credits [options] <cluster_name> <org_name>
Set org cluster credits to given valueneuro admin add-org-cluster-credits [options] <cluster_name> <org_name>
Add given values to org cluster balance
Manging access of org members to clusters:
neuro admin get-cluster-users --org <org_name> <cluster_name>
List all members of orgs added to clusterneuro admin add-cluster-user --org <org_name> <cluster_name> <username>
Add org member to clusterneuro admin remove-cluster-user --org <org_name> <cluster_name> <username>
Remove org member user from the cluster.neuro admin get-user-quota --org <org_name> <cluster_name> <username>
Get info about org member quota in given clusterneuro admin set-user-quota [options] --org <org_name> <cluster_name> <username>
Set org member quota in cluster to given valuesneuro admin set-user-credits [options] --org <org_name> <cluster_name> <username>
Set org member credits in cluster to given valueneuro admin add-user-credits [options] --org <org_name> <cluster_name> <username>
Add given values to org member balance in cluster (#2449)
-
Option
-j
/--jobs
inneuro admin set-user-quota
is now required. Passunlimited
for setting no limit. Negative values are now rejected. (#2453) -
Option
-c
/--credits
inneuro admin set-user-credits
andneuro admin add-user-credits
is now required. Pass "unlimited" inneuro admin set-user-credits
for setting no limit. Values "infinity", "nan" etc are now rejected. (#2454) -
Added support of organizations.
Current organization is displayed in
neuro config show
. It can be changed usingneuro config switch-org
. To least organizations you have access to in each cluster, useneuro config get-clusters
.Also you can run job on behalf of organization. By default,
neuro run
will use current organization, but you can override it usingneuro run --org <some_org>
. (#2465)
- Fix command description for
neuro admin show-cluster-options
. (#2434)
- Fix
neuro rm storage:...
command
- Add Python 3.10 support. (#2421)
- Drop Python 3.6 support. (#2421)
- Configure version checker settings by plugins. (#2405)
- CLI startup time is 2 times shorter now. (#2417)
- Deprecate
neuro project init
command, usecookiecutter gh:neuro-inc/cookiecutter-neuro-project
instead. (#2418)
- Add support of symlinks in
neuro ls
. (#2400)
- Fix sharing buckets using
neuro acl grant
. (#2414)
- Report job price in
neuro job status
, add support of organisation names in jobs. (#2404) - Raise dedicated
NotSupportedError
for unsupported REST API calls (#2407)
-
Added support of open stack as bucket provider. (#2371)
-
Added
blob du <bucket_id_or_name>
command to change bucket's storage usage. (#2372) -
Added support of bucket name in uris for bucket that do not belong to current user. (#2385)
-
Added first name, lastname, registered columns to
neuro admin get-cluster-users
output. (#2387) -
Added "Spent credits" to add "neuro admin get-cluster-users". Splitted 'quota' into 'balance' and 'quota':
neuro admin set-user-quota
changes onlymax_running_jobs
quota.neuro admin set-user-credits
changes onlycredits
balance of user.neuro admin add-user-credits
updatescredits
balance of user by delta. (#2391)
- Fix fetching zero-size blob from S3 compatible bucket providers. (#2394)
- Added support of blob storage for azure clusters. (#2284)
- Added support of blob storage for gcp clusters. (#2293)
- Added
--read-only
flag toneuro blob mkcredentials
to allow creation of read-only credentials. (#2295) - Added
neuro blob importbucket
command to import external bucket. External buckets support the same operations (neuro blob ls/cp/rm/glob
), but it is impossible to generate persistent credentials for them using "neuro blob mkcredentials". (#2297) - Added
neuro blob sign-url <blob-uri>
command for generation of urls that grant temporal access to blob. (#2299) - Added command
neuro blob set-bucket-publicity <bucket> <"public"|"private">
to make bucket accessible for unauthorized users. (#2306) - Undeprecate
neuro job save
command, we decided to support this functionality in future. (#2336)
- Revert back (#2274);
pip install neuro-cli[awscli,boto3]
did not work well.
-
Implement URI autocompletion for "blob:" scheme. (#2273)
-
Provide
pip install neuro-cli[awscli,boto3]
extra dependencies for installing the compatible AWS CLI version.-
neuro-cli[awscli]
installsawscli
package. -
neuro-cli[boto3]
installsboto3
package. -
neuro-cli[awscli,boto3]
installs both. (#2274)
-
- Add support of minio provider as a bucket blob for onprem clusters (#2271)
- Relax aiobotocore requirement; it allows working with the latest aiobotocore without publishing new Neuro SDK. (#2270)
-
Add option
--since
in commandneuro logs
. Add parameter since in jobs.monitor(). (#1964) -
Add option
--timestamps
in commandneuro logs
. Add parameter timestamps in jobs.monitor(). (#2252) -
Implement public URI helpers. (#2253)
-
Reworked blob storage support:
Bucket management commands:
- use
neuro blob mkbucket
to create new bucket - use
neuro blob lsbucket
to list your buckets - use
neuro blob statbucket <bucket_id>
to retrieve info about single bucket - use
neuro blob rmbucket <bucket_id>
to delete bucket. Note that you can only delete empty buckets.
Users can name buckets objects. The name should be unique between users buckets and can be used instead of bucket id.
Bucket contents management commands:
- use
neuro blob ls blob:<bucket_id>/<path_in_bucket>
to list bucket contents. Option-r
disables file system emulation and displays all keys that start with <path_in_bucket>. - use
neuro blob glob blob:<bucket_id>/<glob_pattern>
to glob search objects in buckets. For example,blob:my_bucket/**/*.txt
pattern will match all.txt
files inmy_bucket
bucket. - use
neuro blob cp <src_uri> <dst_uri>
to copy data from/to bucket. - use
neuro blob rm blob:<bucket_id>/<path_in_bucket>
to delete elements from bucket. (#2258)
- use
-
Implement URI autocompletion for bash shell. (#2259)
-
Added way to create persistent credentials to buckets to use in outside of the platform:
- use
neuro blob mkcredentials <bucket1_id> <bucket2_id> ...
to create new credentials for specified buckets - use
neuro blob lscredentials
to list your buckets credentials - use
neuro blob statcredentials <credentials_id>
to retrieve info about single bucket credentials - use
neuro blob rmbucket <credentials_id>
to delete bucket credentials.
Users can name buckets credentials. The name should be unique between users buckets credentials and can be used instead of bucket credentials id. (#2264)
- use
- Deprecate
neuro run IMAGE CMD
, useneuro run IMAGE -- CMD
instead. The same forneuro exec
command. (#2260) - Remove deprecated options older than 6 months. (#2262)
-
Added possibility to disable PyPi version check via environment variable 'NEURO_CLI_DISABLE_PYPI_VERSION_CHECK'. (#2237)
-
Add parameter separator in jobs.monitor(). (#2239)
-
Support URI autocompletion in storage commands for ZSH shell. (#2248)
neuro save
command is deprecated and will be removed in future CLI release. (#2249)
- Fix config permission bits for --pass-config mode (#2233)
- Drop legacy code that puts configuration files for pass-config on storage. (#2233)
- Support VCD cluster config in generate-cluster-config command. (#2193)
- Enabled writing of full logs for each CLI run to a file. Logs are generated under
~/.neuro/logs
folder and automatically removed after 3 days. (#2211) - Added
--cluster
option toneuro run
command andneuro disk
,neuro secret
command groups. This option allows to perform actions for a specific cluster instead of current one. (#2225)
_Admin.set_user_quota
method is fixed for cases, when the amount of credits equals to zero. (#2229)- Recover cursor after an abnormal exit (Ctrl-C). (#2231)
-
All asynchronous iterators returned by API support now an asynchronous manager protocol. It is strongly preferable to use "asyn with" before iterating them. For example::
async with client.jobs.list() as jobs: async for job in jobs: print(job.id) ([#2192](https://github.com/neuro-inc/platform-client-python/issues/2192))
-
Added
neuro storage df
command that allows to retrieve cluster's storage disk usage. (#2201)
- Removed additional '\n' from output of commands that print tokens. (#2200)
- Introduce NDJSONError exception, raise it instead of bare Exception in case of error in ndjson stream. (#2187)
- Preserve volumes order in
client.parse.volumes(...)
call. (#2183)
- Expose
neuro_sdk.EnvParseResult
andneuro_sdk.VolumeParseResult
names. (#2177)
- Dropped
role
argument fromneuro service-account create
: platform automatically generates new role for service account without any permissions. (#2167)
- Add missing requirement to
packaging
. (#2160)
-
Add support of
--owner
and--name
options toneuro image ls
. (#2104) -
Add option
--cluster
in commandsneuro ps
,neuro top
andneuro images
. (#2125) -
Removed support of old runtime quota:
neuro config show-quota
shows credits and max parallel jobsneuro admin add-user-quota/set-user-quota
only support credits and max parallel jobs- Added new command
neuro admin get-user-quota
to print user quota neuro admin get-cluster-user
now prints table with quota info for each user (#2140)
-
Added
neuro service-account
command group. The service account allow to create auth token that can be used for integration with third-party services.neuro service-account create --name optional-name ROLE
creates new service accountneuro service-account ls
lists service accountsneuro service-account get ID_OR_NAME
retrives single service accountnuero service-account rm ID_OR_NAME
removes service account and revokes token. (#2147)
-
Ignore files in parent directories are now used when upload a directory to storage or blob storage. (#1901)
-
Add spinners to the some commands that take time to execute. (#2105)
-
All job related commands support now jobs on other clusters. Commands
neuro exec
,neuro port-forward
,neuro logs
andneuro job save
only support jobs on other clusters if they are specified by URI (jobs on the current cluster can be also specified by bare ID or name). (#2116)
-
Provide forward compatibility with click 8.0 (#2126)
-
Fix hints for 'neuro run' command. (#2127)
-
Fix compatibility with click==8.0. (#2126)
-
Commands
neuro ps
andneuro top
support now outputting several fields in one table cell. By default fields ID and NAME and fields STATUS and WHEN are merged. This saves horisontal space and improves output of long fields (e.g. COMMAND, DESCRIPTION or TAGS). (#2062) -
Added automatic re-attach if container is still alive after
neuro run
/neuro attach
disconnects. (#2085) -
Added warning about actual privilege level if
neuro grant
grants privilege less then user already has. (#2092)
- Fixed unexpected behavior in
neuro admin update-resource-preset
when working with existing preset: Now there are two commands:neuro admin add-resource-preset
to add new preset, options use default values if not specifiedneuro admin update-resource-preset
to update existing preset, options use previous values if not specified Also droppedcluster_name
argument to fix bug when presets were overwritten by active cluster presets. (#2091)
- Added option
--share
inneuro job run
. It allows to share a created job with specified user or role. (#2079)
- Fixed handling jobs with
--pass-config
inneuro job generate-run-command
. (#2074)
- Bump pinned rich version to >=10.0.1 (#2072)
-
Allow bumping jobs life span using
neuro jobs bump-life-span <timedelta>
. (#2020) -
Make a job status reason always printed by neuro status if it is available. (#2047)
-
Add support of automatic re-login after config errors. This allows to upgrade to a new CLI version without manual re-login. (#2048)
-
Made more precise datetime formatting for neuro status: now hours and minutes are always printed. Seconds are printed for event that happened in the last hour. (#2053)
-
Storage operations support now URIs with cluster names different from the current cluster name. You can now list or copy to/from other cluster's storage without switching the current cluster name. (#2054)
-
Added printing of ids of removed disk in
neuro disk rm
. (#2056) -
Allow to specify date and time in --since/--until relatively to current moment. For example,
neuro ps --since 1d
will show jobs that were created during last day. (#2059) -
Image operations support now URIs with cluster names different from the current cluster name. (#2066)
-
Added hint about headless login when
neuro login
executes in non-GUI environment. (#2068)
-
Do not add implicit '\n's to
neuro job generate_run_command <job-id>
command output. (#2034) -
Fix bad formatting in
neuro run
command. (#2055) -
Do not kill a non-tty job when running in non-tty mode automatically. This allows to exit from
neuro run ... | tee
using^C
without killing a job. (#2063)
-
Added option
--format
inneuro top
for specifying the output columns. It is similar to option--format
ofneuro ps
, but support several additional columns. (#2000) -
Added filtering options in
neuro top
:--owner
,--name
,--tag
,--description
,--since
and--until
. They are similar to corresponding options inneuro ps
. By default shown only jobs of the current user, specify--owner=ALL
to show jobs of all users. (#2001) -
Add option
--sort
inneuro top
. (#2002) -
Drop cluster_name and description from default output of
neuro ps
command. (#2009) -
Allow deletion images without tag. Support multiple arguments for
neuro image rm
command. (#2010)
- Support job statuses "suspended" and "unknown" in
neuro ps
andneuro top
. (#2011)
- Fixed parsing of huge job payloads. (#2004)
-
Added support of named disks:
- create new disk by
neuro disk create --name <disk-name> STORAGE
command - name can be used to get/delete disk:
neuro disk get <disk-name>
orneuro disk delete <disk-name>
- name can be used to mount disk:
neuro run -v disk:<disk-name>:/mnt/disk ...
(#1983)
- create new disk by
-
Added printing of job id when
neuro flow
orneuro attach
exits. (#1993) -
Added "Life span ends" field to
neuro status <job-id>
. (#1994) -
Now all datetime fields are printed in "humanized form". Use
neuro --iso-datetime-format <command> ...
to force ISO 8601 format. (#1995) -
Added recovering of attachment to job after internet connection issue in
neuro attach
andneuro run
. (#1996)
-
neuro top
supports now multiple jobs. If multiple jobs is specified at command line, it outputs a table for that jobs. If no jobs are specified, it outputs all active user's job. (#418) -
Added private option
--x-trace-all
which forces distribute tracing. (#1973) -
Add
neuro job generate-run-command <job-id-or-name>
to simplify reruning of jobs. (#1977) -
Added new options to
neuro ps
:--distinct
: Show only first job if names are same.--recent-first/--recent-last
: Show newer jobs first or last (#1981)
- Soft-deprecate
neuro job tags
command. (#1840)
-
Removed support of deprecated
--status=all
inneuro ps
. Use--all
instead. (#1883) -
Refresh config after resource presets are updated. (#1899)
-
Don't open browser on
login-with-token
. (#1748) -
Suppress non-critical neuro configuration database errors. (#1816)
-
Fixed unintentional interpretation of sequences like
:+1
and[b]
in descriptions, commands, paths, URIs, etc when output on the console. (#1917)
- Fix unstable imports of
rich
tool. (#1911)
- Extract
neuromation.api
intoneuro-sdk
package, renameneuromation
toneuro-cli
. (#1892)
-
Generating the URI string for
RemoteImage
now correctly escapes special characters like "#", "%", "?" in image name and tag. (#1895) -
Fixed conflict between logging and showing progress in
neuro cp -vv
. Use therich
library for outputting colored logs. (#1897)
-
Added --privileged flag to
neuro run
. Added corresponding argumentprivileged
tojobs.start
. (#1879) -
Added Disk life-span information to the
neuro disk ls --long-format
andneuro disk get
commands. (#1880)
- Added support of filtering by statuses "cancelled" and "suspended" in
neuro ps
. (#1881)
-
Add
-l
option toneuro image tags
for long output format (#1855) -
Add
-f
flag toneuro image rm
to force delete images that have multiple tag references (#1828) -
Enable display of non-zero restarts count for all jobs (previously only jobs with proper --restart-policy had this field). (#1859)
-
Added cluster resource presets management commands. Added
Preemptible Node
column in resource presets format inneuro config show
. (#1863) -
Pass preset instead of container resources to job api. (#1864)
-
neuro blob cp
uses now partial read when retry to download a blob. (#1865) -
Change jobs capacity API endpoint. (#1872)
-
Add disk column to
admin get-clusters
command. (#1873) -
Added options
--update
and--continue
for commandneuro blob cp
. Added optional boolean parametersupdate
andcontinue_
to the corresponding API. (#1875)
- Fixed parsing file modification time in Blob Storage. (#1874)
-
Moved pass_config option to API. Now it uses config completely encoded into ENV variable. (#1814)
-
Added support of max running jobs quota. The
neuro admin set-user-quota -j <count>
command configures this quota for user. By default, a new job cannot be created after the quota is reached, but the--wait-for-seat
flag allows creating a job that will wait for another job to stop. (#1827) -
When the connection is lost during transferring files
neuro cp
retries now only sending and retrieving new data instead of starting file operation from the start. Added optional parameters offset and size in methodstorage.open()
for partial reading. Added methodstorage.write()
for overwriting the part of the file. (#1831) -
Added option
--continue
for commandneuro cp
. It specified, copy only the part of the source file past the end of the destination file and append it to the destination file if the destination file is newer and not longer than the source file. Otherwise copy the whole file. Added corresponding keyword-only boolean parametercontinue_
to the API. (#1841) -
Add support of
SUSPENDED
job status. (#1844) -
Added support of
neuro ps --owner ME
. This allows to filter current users jobs. (#1845) -
The active cluster name can now be specified in local configuration file. (#1846)
-
Add
neuro image rm
command for removing images from remote registries (#1770) -
Allowed to share disks. (#1811)
-
Added support of multiple disk removal:
neuro disk rm disk-id-1 disk-id-2
works properly now. (#1821) -
Added displaying of restarts count for restartable jobs. Added
restarts
field toJobStatusHistory
. (#1822)
- Bump
aiohttp
to >= 3.7.2 in the library requirements, drop transient dependenciesmultidict
andyarl
. Support Python 3.8 and Python 3.9 (#1802)
- Allowed to share secrets. (#1791)
- Made
neuro completion patch
idempotent. (#1760)
- Suppress connection errors when cluster is not available in
neuro config show
. (#1763)
-
Fragment, query, user, password and port parts are not allowed now in URIs (these parts were silently ignored before). Fixed support of local file paths containing characters like "#", "%", ":", "?", "@" and "~". (#1531)
-
Implemented disks management commands. (#1716)
-
neuro run
allows now to specify disk volumes using-v disk:<DISK>:<PATH>:<RW_FLAG>
. (#1721) -
Added support of
created_at
andlast_usage
field of disks. Added--long-format
option toneuro disk ls
. (#1729) -
Memory amount parsing now supports of both
b
andB
suffixes for specifying decimal quantities. Improvedneuro disk create
docs. (#1731) -
Added a
--life-span
argument toneuro disk create
. Added alife_span
argument inclient.disks.create()
. (#1734) -
Added output to
neuro disk rm
andneuro secret rm
when-v/--verbose
global flag is set. (#1738) -
Quite mode command
neuro -q disk ls
now prints simple list of disk ids. (#1740) -
Command
neuro image tags
outputs now a list of tags instead of a list of images. (#1741) -
Added support of
--full-uri
toneuro disk get
. (#1747) -
Show a deprecation warning for
--life-span=0
inneuro run
command. (#1749) -
Change the color scheme for job statuses. The yellow color is for the cancellation, cyan is for pending jobs. (#1752)
-
Commands
neuro image ls
andneuro image tags
and corresponding APIimages.ls()
andimages.tags()
can now return more than 100 items. (#1606) -
Make
neuromation.api.SecretFile
class public (#1714)
-
Added support of
CANCELED
state. (#1696) -
Added support of error messages in streamed delete response. (#1697)
-
Colorize
neuro ps
output. (#1698) -
Added way to destroy browser session by during
neuro logout
. (#1699) -
Remove
--volume=ALL
option from CLI and move volume and env variable parsing from CLI module to theParser
class in API (#1707) -
Command
neuro acl list
accepts now a URI prefix for filtering. API functionsusers.get_acl()
andusers.get_shares()
accept now the uri argument. (#1708) -
neuro run
supports now multiple--env-file
options. (#1710)
-
Show add available jobs counts (cluster capacity) in
neuro config show
command. (#1687) -
Make JobStatus calculation forward-compatible;
JobStatus.UNKNOWN
is returned for unknown statuses but the code doesn't raiseValueError
at least. (#1688)
- Pin
yarl
version dependency to 1.5.1+.
-
Implement
FileStatus.uri
property. (#1648) -
Add support of plugin-defined config parameters (#1657)
-
Added
find_project_root
function (#1660) -
Added
neuro rm --progress
andprogress
argument toStorage.rm
for tracking delete progress (#1664) -
Added
internal_hostname_named
toJobDescription
and to output ofneuro job status
. (#1675) -
Added logging of
X-Error
error description inneuro port-forward
. (#1676) -
Added printing of documentation link to
nuero login
command (#1680) -
Added
neuro admin show-cluster-options
command for displaying possible cluster configuration options. (#1681)
-
neuro ps
supports now columns "created", "started" and "finished" (hidden by default). (#1020) -
neuro status
shows now the job's http port. (#1375) -
Long list of tags for
neuro status
is now wrapped. (#1408) -
neuro ps
supports now the "life_span" column (hidden by default). (#1448) -
Command aliases are now supported if not logged in. (#1480)
-
Added support of the
--schedule-timeout
option inneuro run
. (#1499) -
Added commands for adding and removing roles:
neuro acl add-role
andneuro acl remove-role
. (#1582) -
Support expansion of the user home directory ("~") in the file path argument in
neuro secret add
. (#1610) -
Put
--help
option first in the help output for a command or command group (#1627) -
Officially support Python 3.8. (#1638)
-
Create a topic about secrets management and usage. (#1640)
-
Fixed "division by zero" error when copy from
/dev/stdin
to storage. (#1129) -
Fixed support of local images whose name starts with Neuro registry prefix, e.g.
registry.neu.ro/alice/imagename
. (#1159) -
Fixed support of relative image URIs with numeric name, like
image:5000
. (#1631) -
When resolve job URI with other user's name and job name (like
job:/bob/test-name
), the owner name no longer ignored when the shared job is not found. (#1633)
- Removed support of
--volume=HOME
inneuro run
. (#1202)
-
Implement secrets management commands. (#1545)
-
neuro run
allows now to specify secrets: either as a file-v secret:<KEY>:<PATH>
or as an environment variable-e <NAME>=secret:<KEY>
. (#1558) -
Support azure cluster config file generation. (#1577)
-
Implement
--port-forward
option forrun
andattach
commands. (#1601) -
Take Secrets Service URL from
/config
. (#1607)
- Deprecate and hide
submit
command. (#1602)
-
Always set "LESS=-R" env variable to fix outputs with scrolling, e.g. "neuro help" (#1595)
-
Fix a warning raised by
neuro job port-forward
command on Python 3.6 (#1592)
- Increase timeout for waiting for jobs exit after finishing attached session from 15 seconds to 15 minutes. (#1584)
-
A
.neuroignore
file specifies files thatneuro cp
should ignore when upload directories to the storage or object storage. Syntax and semantic are similar to.gitignore
files. (#1446) -
API:
Client.jobs.run()
supports now relative storage URIs for volumes. (#1464) -
Explicit options
--exclude
and--include
no longer disable the defaults from the storage.cp-exclude configuration variable. Use explicit--exclude="*" or --include="*"
to override default patterns. (#1489) -
Support leading and trailing slashes (
/
) in filters. Leading slash prevents matching in subdirectories, e.g./*.txt
matchesspam.txt
, but notdir/spam.txt
, while*.txt
matches both of them. Patterns with trailing slash match only directories, while patterns without trailing slash match both directories and files. (#1490) -
Implement attach/exec/interactive-run using WebSockets. (#1497)
-
Implement port forwarding over WebSockets. (#1555)
-
Added the
--exclude-from-files
option inneuro cp
.API: Added the optional parameter
ignore_file_names
in theupload_dir()
methods ofstorage
andblob_storage
. (#1560)
- Fix swallowing underscores in
neuro status
. (#1565)
- Drop deprected and hidden
neuro storage load
command. (#1542)
- Suppress annoying message about improperly handled GeneratorExit exception. (#1525)
-
API:
Storage.ls()
is an asynchronous generator now. (#1457) -
Added the
--restart
option toneuro run
andneuro submit
and API. (#1459) -
API:
Jobs.list()
is an asynchronous generator now. (#1473) -
neuro ps
now outputs local date instead of UTC and interprets--since
and--until
options as local time if the timezone is not specified. (#1477) -
autocomplete job name/id (#1485)
-
Unencodable characters are now replaced with
?
orU+FFFE
when output to the stdout. (#1502)
-
Fixed downloading a file when it is restarted for some reasons. The newly downloaded data no longer appended to a previously downloaded data, but overwrites it. (#1521)
-
Fixed parsing image URIs with the single slash after scheme, like
image:/otheruser/imagename
. (#1505)
- Added
--since
and--until
options toneuro ps
. (#1461)
- Make exit code of
job run
command more reliable. (#1470)
-
Commands
neuro ps
,neuro job status
,nauro image list
andneuro acl list
output now URIs in the short form if possible. Use the new--full-uri
option to get full qualified URIs. (#1330) -
Changed interpretatation of cluster related URIs (with schemes
storage:
,image:
andjob:
) with missed host and path started with/
.storage:/user/path
is expanded now tostorage://{defaultcluster}/user/path
, so you do not need to specify the cluster name when refer to other user's resources on the same cluster. (#1424) -
Implement
neuro storage tree
command for displaying the directory tree on storage. (#1435) -
Filter patterns are now more compatible with
.gitignore
. Added support of**
which matches zero or more path components.?
and*
no longer match/
. Patterns which does not contain/
at the beginning or middle matches now files in subdirectories. (#1444)
- Bump
typing_extensions
dependency version to satisfy CLI requirements. (#1421)
- Fix
--pass-config
error: File exists: '/root/.neuro'. (#1415)
-
Support job run-time limit:
neuro run --life-span "1d2h3m"
. (#1325) -
For cluster-specific resources (with schemes
storage
,image
andjob
) client now use URIs containing the cluster name, e.g.storage://{clustername}/{username}/{path}
instead ofstorage://{username}/{path}
. Relative URIstorage:{path}
will be expanded to absolute URI containing the current cluster name and the user name:storage://{clustername}/{username}/{path}
. Same forimage
andjob
schemes. (#1341) -
Added 'neuro image ls -l' option, which also prints Docker URL-s (#1354)
-
Added the
-u
option inneuro acl list
to specify a role or user to which resources are available and for which they are shared. (#1355) -
Sort
neuro admin get-cluster-users
by name. (#1359) -
Steal config files on
neuro job --pass-config
. (#1361) -
Support hidden files on the storage. Hide names started with a dot by default, provide
neuro ls --all
option to show all files. (#1362) -
Add HTTP tracing of neuro login commands. (#1387)
-
Support job tags:
neuro run --tag=experiment-1
,neuro ps --tag=experiment-1
. (#1393) -
Support job tags listing:
neuro job tags
. (#1396) -
Optionally display job's tags in
neuro ps
(the feature needs to be explicitly enabled in.neuro.toml
config file). (#1406)
-
Fix the alias finding routine when user is not logged in. (#1360)
-
neuro kill
exits now non-zero code if it failed to kill any job in the list. (#1272) -
Support un-quoted commands for neuro-exec:
neuro exec bash -c "ls && pwd"
is now a valid syntax. (#1321)
-
Support custom columns format for
neuro ps
command. (#1288) -
Support custom aliases. (#1320)
-
Removed support of
~
in URIs (likestorage://~/path/to
). Relative URIs can be used instead (likestorage:path/to
). Support of tilde in local file paths (like~/path/to
) has been preserved. (#1329)
-
--env-file
now allows blank lines and comments (lines starting with "#") in the file. (#1208) -
Send the usage statistics to Google Analytics (#1286)
-
Use Sqlite for saving the
~/.neuro/db
configuration file. (#1298)
-
Fix columns width and aligning for
neurp ps --format=<>
command. (#1302) -
Fix ps-format documentation issues. (#1303)
-
Parameters specified in ps-format now always override default values.
width
takes priority overmin
andmax
,max
takes priority overmin
. (#1310)
-
Add
neuro admin add-user-quota
andneuro admin set-user-quota
commands to control user quotas (#1142) -
Added options
--exclude
and--include
inneuro storage cp
. (#1182) -
Adjust NAME column of
neuro ps
so that grep by name works. (#1189) -
Read
neuro ps --format
spec from config files if present. (#1294) -
Read
neuro cp
filters from user configuration file. (#1295)
-
Now
neuro images
do not require the installed Docker. (#1071) -
Implement
neuro config get-clusters
command. (#1177) -
Convert configuration file into configuration directory, now it is
~/.neuro
folder. (#1183) -
Implement
neuro config switch-cluster
for switching between available clusters. (#1217) -
Implement
neuro admin get-clusters
command. (#1223) -
Implement
neuro admin add-cluster
command. (#1224) -
Add
neuro admin generate-cluster-config
command. (#1227) -
neuro project init
now supports argument to set default value for generated project directory (#1230)
- Correctly process both quoted command arguments (
neuro run python:latest "python3 -c 'import os'") as well as unquoted version (
neuro run python:latest python3 -c 'import os'`). (#1229)
-
Trace sent HTTP requests and received replies to stderr if
--trace
flag is passed. (#467) -
Display
Cluster
field for job status and listing commands. (#874) -
Display
Entrypoint
field for job status command. (#924) -
Display volumes information for
neuro status
command. (#1003) -
Option
--volume=HOME
deprecated. (#1009) -
Provide client.presets property, update docs. (#1078)
-
Retry storage operations in case of some errors. (#1107)
-
neuro kill
will continue work if multiple jobs specified but you haven't required permissions for some of them. (#1122) -
Introduce
neuro config show-quota
. (#1141) -
Use pager for long lists. (#1152)
-
Add global option '--hide-token/--no-hide-token' to be used together with '--trace' for preventing the user's token from being printed to stderr for safety reasons. (#1158)
-
Suppress security checks for config files if NEUROMATION_TRUSTED_CONFIG_PATH environment variable is on. (#1173)
-
When running a job with
--detach
optionneuro
now returns an error status if job fails to start at all (e.g., when cluster scale up fails). If job starts successfully (regardless of its run result)neuro run
with--detach
returns 0 like before. (#1059) -
Provide default arguments for api.Resources constructor to keep broken backward compatibility. (#1070)
-
Fix help message for
neuro project init
: in fact, the command does not accept an argument. (#1080) -
Process 502 Bad Gateway as a separate exception, don't miss it with 400 Bad Request. (#1111)
-
Wait for
ThreadPoolExecutor
finish befor exit from the program. (#1144)
-
Introduce
neuro project init
for scaffolding an empty project. (#1043) -
neuro cp -r
now works with non-directories. (#1053) -
Disable logging of annoying SSL errors. (#1065)
-
Make image commands work on Windows. (#1067)
-
Fix bug with job-name resolution introduced in release 19.8.19: do not print annoying warning messages if more than one job with specified name was found. (#1034)
-
Fix
neuro job save
timing out too early. (#1062)
- Improved displaying a progress of copying many files. (#1034)
- Deprecate
neuro storage load
command. (#1028)
-
Due to
certifi
update delays, show certifi package error not earlier than two weeks after the latest version was released. (#944) -
Support streaming output for
neuro-save
command. (#946) -
Implement mounting all available volumes:
neuro run --volume=ALL
. (#974) -
Uploading/downloading of a directory containing many small files is now 4-10 times faster. (#981)
-
Support Cloud TPU: new cli options
--tpu-type
,--tpu-sw-version
added. (#982) -
Support job operations via job-URI (e.g.,
neuro status job://owner-name/job-name
). (#988) -
Support job filtering by owner:
neuro ps -o user-1 --owner=user-2
. (#990) -
Added the
--update
option inneuro storage cp
. It makes the command copying the source file only when it is newer than the destination file or when the destination file is missing. (#1007) -
Added the
-d
(--directory
) option inneuro storage ls
which makes the command to list directories themselves, not their contents. (#1012)
FileStatus.permission
now isAction
, wasstr
. (#963)
- Fix regression: restore port-forward functionality. (#979)
-
Change default interactivity option for
neuro exec
: by default, interactive--tty
will be used. (#942) -
Use
Optional[datetime]
inJobStatusHistory
.created_at
,.started_at
,.finished_at
instead ofstr
. (#955) -
neuro storage cp
is now up to 2 times faster for a directory with many small files. (#958) -
Refactor port forwarding api to use async context manager for controlling a time of port forwarding process. (#959)
- Support remote images with registry
host:port
. (#939)
-
The
-p/--non-preemptible
parameter forneuro run
has been removed in favor in embedding it into presets coming from the server. (#928) -
Show progress for
neuro cp
by default. (#933) -
Use dataclasses in image progress API (#935)
-
Add option
ps --all
, deprecateps -s all
. (#538) -
Forbid mixing arguments and CLI options in
run
,submit
andexec
commands. Options (parameters starting from dash or double dash, e.g.-n
and--name
) should prepend arguments (e.g.image:ubuntu:latest
). All tailed options belong to executed container command, not to neuro CLI itself. (#927)
-
The behavior of
neuro storage mv
is now closer to the behavior of themv
command. It moves now files inside the target directory if it exists and moves a file under the new name otherwise. Added also options--target-directory
(-t
) and--no-target-directory
(-T
). (#203) -
neuro job submit
andneuro job run
command now support--pass-config
option. This option ensures your local neuromation config file is uploaded to your job allowing you to executreneuro
commands from the container. (#827) -
Add
neuro image tags
command to list all tags for a given image (#852) -
Stabilize jobs API (#879)
-
neuro storage cp
,neuro storage mv
andneuro storage rm
support now globbing for source arguments. Globbing can be disabled by specifying new option--no-glob
. Added commandneuro storage glob
which lists resources matching patterns. (#889)
-
Implemented
neuro job browse
. Added the--browse
flag toneuro job submit
andneuro job run
. (#571) -
Added the global
--quiet
option, opposite to--verbose
. Both options are additive. The--quite
options forneuro job
andneuro image
are deprecated now. (#848) -
Drop
neuromation.api.NetworkPortForwarding
for the sake ofneuromation.api.HTTPPort
(#861) -
The output of "storage" commands in verbose mode is now more similar to the output of corresponding POSIX command. In particular
neuro -v storage cp -r
outputs a line for every copied file or directory. (#869) -
The behavior of the
neuro storage cp
is now closer to the behavior of thecp
command. It now copies files inside the target directory if it exists and copies a file under the new name otherwise. Added also options--target-directory
(-t
) and--no-target-directory
(-T
). (#870)
- Fix certifi upgrade suggestion text. (#845)
- Make non-preemtible mode default. (#829)
- Improve storage operations stability by supporting sticky sessions. (#832)
- Relax pyyaml version requirement to don't force users to upgrade it if pyyaml was installed by anaconda or another non-pip installer. (#828)
- Don't run version checks if config is not loaded by CLI command. (#826)
-
Changes in
neuro store mkdir
behavior: fails if the directory already exists or parent directories do not exist. Add option--parents
to make parent directories as needed and ignore existing directories. (#131) -
Changes in
neuro store rm
behavior: removes only files by default, add option--recursive
to remove directories. (#354) -
Storage commands (
neuro storage ls
,neuro storage cp
,neuro storage mv
,neuro storage rm
) accept now variable number of arguments. This allows to copy or remove several files by one command. (#784) -
Implement
neuro config login-headless
command. (#793) -
Changes in interpretation URIs with absolute path and without host name.
storage:///foo/bar
means now the same asstorage://foo/bar
instead ofstorage://{currentuser}/foo/bar
, andstorage:///
can be used for access to the storage root directory. (#808) -
neuro storage cp
now supports copying to/from non-regular files like character devices and named pipes. In particular this allows to output the file to the stdout or get the input from the stdin (/dev/stdout
and/dev/stdin
on Linux,CON
on Windows). (#813) -
Relax certifi required version. Raise a warning if the package should be updated. (#819)
- Allow to logout even if config file is broken. (#792)
-
Print exposed HTTP for named jobs (#736)
-
Support retrieving server config for authorized users. (#766)
- Implement job run command. (#652)
- Fix image name parser to substitute lastest tag automatically. (#729)
-
New option
--neuromation-config
for using alternative config file location. Environment variableNEUROMATION_CONFIG
can be used as option. New commandneuro config docker
allows to usedocker image push/pull
commands with platform registry. (#381) -
neuro port-forward
command now accepts multiple local-remote port pairs in order to forward several ports by a single command. (#632) -
Support job names. (#648)
-
Make progress argument for storage API optional. (#687)
-
Rename neuromation.client to neuromation.api (#688)
-
Implement
neuro config login-with-token TOKEN URL
command. (#712)
-
Don't allow to submit image names starting with dash. (#526)
-
Respect
--network-timeout
option inlogs
andcp
operations. (#703)
- Remove deprecated functionality:
neuro model
,neuro config id_rsa
andneuro job submit --ssh
option. (#700)
- Change the default API endpoint to
https://staging.neu.ro/api/v1
(#666)
- Fix incorrect
--volume
parsing under windows (#635)
- New flag
--http-auth/--no-http-auth
which controls authentication for forwarded HTTP port. Enabled by default. (#604)
-
Increase wait time to finish on POSIX platforms. It dignificantly decreases a chance of error report after CLI command executions. (#597)
-
Fix forward compatibility with platform server. (#599)
-
Close version checker properly in case of error in a command execution. (#586)
-
Pin aiohttp to 3.5+ to satisfy minimal supported CLI version. (#587)
-
Extended '/dev/shm' space (
--extshm
) turned on by default (#449) -
Add support for server-side job list filtering (speedup of the
neuro ps
) (#474) -
Several UX improvements. (#486)
-
neuro store ls
now supports colored output, LS_COLORS(GNU) or LSCOLORS(BSD) environment variable required. (#487) -
Improve shell completion logic, make shell type argument mandatory. (#506)
-
Add Http URL to neuro job submit output (#527)
-
Added neuro port-forward command to forward a port of a job exposed during job submit to a local one. (#535)
-
Support Windows platform (#548)
-
Check the lastest PyPI neuromation release, suggest to upgrade if PyPI has a newer version. (#308)
-
Changes in
neuro store ls
behavior: display files by columns by default, add option-l
for long output, display one per line for pipes by default. (#427) -
Set up the platform to work with the new platform DNS names. (#495)
-
Use colored mode only if all stdin, stdout, and stderr are tty. (#473)
-
Improved login error reporting (#477)
-
Add
-p/-P
shortcuts for--preemtible/--non-preemtible
forneuro submit
command. (#458) -
Wait for job start/failure on job submit. (#356)
-
Support
-v
as a shortcut for--volume
inneuro submit ...
command. (#383) -
Improve resource URI normalization and print normalized URIs back to the user. (#457)
-
Re-organize CLI commands for better UI/UX experience. Obsolete commands are hidden but still supported. (#460)
-
Add top-level aliases for the most frequent commands. (#439)
-
Better formatting for examples section. (#446)
-
Bump
click
version to7.0+
. (#437) -
Temporary disable scary logging about unhandled exception. (#438)
-
Fix an error in local path normalization. (#443)
-
Implement
neuro job top
(#412) -
Pretty format output for
neuro config show
command, print current authentication token byneuro config show-token
. (#426) -
Check
~/.nmrc
config file for strict0o600
permissions. (#430)
- Drop
--token
and--url
parameters from the root command, useneuro login
andneuro config url
to setup config parameters. (#430)
-
Deep refactor
neuro store
command and corresponding API client. (#324) -
Default API URL switched to HTTPS for neuro cli (#325)
-
Job resource output formatting for command
neuro job status
changed (#328) -
neuro image pull/push
command improved for support different names/tags for images, introduced 'image://' scheme for image referencing. (#349) -
Implement --show-traceback command line option to show python traceback in case of top-level error (#365)
-
Added new option
--insecure
for storing auth in plain text file instead system keyring. (#366) -
New
neuro image ls
command for listing custom images available on platform repository. (#367) -
Added new command
neuro job exec
to execute commands in already running job without ssh server. (#373) -
Display Preemptible in job status output (#393)
-
Make the client work on Python 3.7 (#402)
-
Implement
neuro job top
(#412)