Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nyxt: 2.2.4 -> 3.1.0 #235095

Merged
merged 6 commits into from
Jun 10, 2023
Merged

nyxt: 2.2.4 -> 3.1.0 #235095

merged 6 commits into from
Jun 10, 2023

Conversation

dariof4
Copy link
Contributor

@dariof4 dariof4 commented May 31, 2023

Description of changes

Update Nyxt from version 2.2.4 to version 3.1.0.
Release notes since 2.2.4:
https://nyxt.atlas.engineer/article/release-3.0.0.org
https://nyxt.atlas.engineer/article/release-3.1.0.org

Fixes: #217888

This also updates the lisp dependencies needed to build Nyxt 3.1.0 and adds prompter which was transferred from Nyxt to an external library not yet on quicklisp, it also changes the build from nyxt/gtk-application to nyxt/gi-gtk-application since that seems to be the one to package according to Nyxt devs (the makefile and Guix package gi-gtk).

I have a few concerns though:

  1. Each time Nyxt is launched it for some reason tries to recompile nasdf (which it fails since it can't create /homeless-shelter) and crashes with an system nyxt out of date error, I found several ways to "fix" this, either setting CL_SOURCE_REGISTRY=$CL_SOURCE_REGISTRY:$(pwd)// in postConfigure which makes it stop trying to recompile itself, using (asdf:disable-output-translations) like stumpwm does it, which makes it still try to recompile nasdf and fails, but makes Nyxt run fine, or setting HOME=/tmp which allows Nyxt to successfully recompile nasdf in /tmp/.cache (Guix sets HOME=/tmp for Nyxt as well, may be related?).
    I'm a little bit out of my depth here and have no clue why Nyxt would be trying to recompile nasdf, it seems to be something to do with CL_SOURCE_REGISTRY, but I don't have enough experience with CL to understand this at all 😅
    I would appreciate if someone more experienced could shed some light as to what is the best solution.

  2. Should this PR be split up into multiple? and if so in how many should it be split? I tried to keep the commits be logically separate so this should be easy.

  3. I think using wrapLisp within Nyxt's default.nix (needed to compile Nyxt with the --dynamic-memory-size flag to increase the heap size) causes quite a few extra builds, since the way I understand it can't use the cache for all the lisp dendencies of Nyxt, since it uses wrapLisp, it might be better to apply --dynamic-memory-size globally to sbcl?

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@dariof4
Copy link
Contributor Author

dariof4 commented May 31, 2023

Result of nixpkgs-review run on x86_64-linux 1

46 packages failed to build:
  • sbclPackages._40ants-doc-test
  • sbclPackages.april-demo_dot_ncurses
  • sbclPackages.bus
  • sbclPackages.cl-clblas-test
  • sbclPackages.cl-feedparser
  • sbclPackages.cl-feedparser-tests
  • sbclPackages.cl-libpuzzle-test
  • sbclPackages.cl-mecab-test
  • sbclPackages.cl-oclapi-test
  • sbclPackages.cl-pslib
  • sbclPackages.cl-pslib-barcode
  • sbclPackages.clack-handler-fcgi
  • sbclPackages.clath
  • sbclPackages.cletris-test
  • sbclPackages.clim-widgets
  • sbclPackages.climon-test
  • sbclPackages.data-frame
  • sbclPackages.dfio
  • sbclPackages.dom
  • sbclPackages.exscribe
  • sbclPackages.gendl
  • sbclPackages.geysr
  • sbclPackages.graphs
  • sbclPackages.gwl
  • sbclPackages.gwl-graphics
  • sbclPackages.hamcrest-tests
  • sbclPackages.ledger
  • sbclPackages.lisp-stat
  • sbclPackages.maiden-dictionary
  • sbclPackages.mockingbird-test
  • sbclPackages.monomyth
  • sbclPackages.nyxt
  • sbclPackages.nyxt-ubuntu-package
  • sbclPackages.pgloader
  • sbclPackages.pkg-doc
  • sbclPackages.plot
  • sbclPackages.prometheus_dot_test_dot_all
  • sbclPackages.regression
  • sbclPackages.robot
  • sbclPackages.ta2
  • sbclPackages.tasty
  • sbclPackages.translators
  • sbclPackages.tree
  • sbclPackages.wire-world
  • sbclPackages.yadd
  • sbclPackages.zaserve
374 packages built:
  • nyxt
  • sbclPackages._40ants-doc-full
  • sbclPackages.april
  • sbclPackages.april-demo_dot_cnn
  • sbclPackages.april-demo_dot_fnn
  • sbclPackages.april-lib_dot_dfns_dot_array
  • sbclPackages.april-lib_dot_dfns_dot_graph
  • sbclPackages.april-lib_dot_dfns_dot_numeric
  • sbclPackages.april-lib_dot_dfns_dot_power
  • sbclPackages.april-lib_dot_dfns_dot_string
  • sbclPackages.april-lib_dot_dfns_dot_tree
  • sbclPackages.april-xt_dot_uzuki
  • sbclPackages.assoc-utils-test
  • sbclPackages.aws-foundation
  • sbclPackages.aws-sdk
  • sbclPackages.birch_dot_test
  • sbclPackages.bit-smasher-test
  • sbclPackages.bnf_dot_test
  • sbclPackages.can-test
  • sbclPackages.carrier
  • sbclPackages.caveman2
  • sbclPackages.caveman2-test
  • sbclPackages.caveman2-widgets
  • sbclPackages.caveman2-widgets-bootstrap
  • sbclPackages.caveman2-widgets-bootstrap-test
  • sbclPackages.caveman2-widgets-test
  • sbclPackages.check-bnf_dot_test
  • sbclPackages.chirp-dexador
  • sbclPackages.ciao
  • sbclPackages.circular-streams-test
  • sbclPackages.cl-amqp_dot_test
  • sbclPackages.cl-annot-prove
  • sbclPackages.cl-annot-prove-test
  • sbclPackages.cl-ansi-text
  • sbclPackages.cl-ansi-text_dot_test
  • sbclPackages.cl-ascii-art
  • sbclPackages.cl-association-rules-tests
  • sbclPackages.cl-base58-test
  • sbclPackages.cl-bson-test
  • sbclPackages.cl-cache-tables-tests
  • sbclPackages.cl-cognito
  • sbclPackages.cl-coinpayments
  • sbclPackages.cl-colors2
  • sbclPackages.cl-cookie
  • sbclPackages.cl-cookie-test
  • sbclPackages.cl-coroutine-test
  • sbclPackages.cl-coveralls
  • sbclPackages.cl-coveralls-test
  • sbclPackages.cl-covid19
  • sbclPackages.cl-darksky
  • sbclPackages.cl-darksky-test
  • sbclPackages.cl-data-structures-tests
  • sbclPackages.cl-debug-print-test
  • sbclPackages.cl-disque-test
  • sbclPackages.cl-dotenv-test
  • sbclPackages.cl-emoji-test
  • sbclPackages.cl-events_dot_test
  • sbclPackages.cl-fixtures-test
  • sbclPackages.cl-gearman-test
  • sbclPackages.cl-gists
  • sbclPackages.cl-gists-test
  • sbclPackages.cl-gopher
  • sbclPackages.cl-haml-test
  • sbclPackages.cl-hash-table-destructuring-test
  • sbclPackages.cl-incognia
  • sbclPackages.cl-ini-test
  • sbclPackages.cl-intbytes-test
  • sbclPackages.cl-jsx-test
  • sbclPackages.cl-kraken
  • sbclPackages.cl-ksuid
  • sbclPackages.cl-ksuid-test
  • sbclPackages.cl-libsvm-format-test
  • sbclPackages.cl-locale-test
  • sbclPackages.cl-ltsv-test
  • sbclPackages.cl-markup-test
  • sbclPackages.cl-mustache-test
  • sbclPackages.cl-notebook
  • sbclPackages.cl-online-learning-test
  • sbclPackages.cl-ply-test
  • sbclPackages.cl-project
  • sbclPackages.cl-project-test
  • sbclPackages.cl-punch-test
  • sbclPackages.cl-random-forest-test
  • sbclPackages.cl-reexport-test
  • sbclPackages.cl-rules-test
  • sbclPackages.cl-selenium
  • sbclPackages.cl-selenium-test
  • sbclPackages.cl-ses4
  • sbclPackages.cl-shellwords-test
  • sbclPackages.cl-slug-test
  • sbclPackages.cl-ssdb-test
  • sbclPackages.cl-statsd_dot_test
  • sbclPackages.cl-strings-tests
  • sbclPackages.cl-tasukete
  • sbclPackages.cl-tasukete-test
  • sbclPackages.cl-telebot
  • sbclPackages.cl-telegram-bot
  • sbclPackages.cl-test-more
  • sbclPackages.cl-textmagic
  • sbclPackages.cl-textmagic-test
  • sbclPackages.cl-toml-test
  • sbclPackages.cl-transmission-test
  • sbclPackages.cl-voxelize-test
  • sbclPackages.cl-weather-jp
  • sbclPackages.cl-weather-jp-test
  • sbclPackages.cl-webdriver-client
  • sbclPackages.cl-webdriver-client-test
  • sbclPackages.cl-webkit2
  • sbclPackages.cl-yaclyaml
  • sbclPackages.cl-yaclyaml-tests
  • sbclPackages.cl-zipper-test
  • sbclPackages.cl_plus_ssl_dot_test
  • sbclPackages.clache-test
  • sbclPackages.clack-handler-woo
  • sbclPackages.clack-handler-wookie
  • sbclPackages.clack-pretend
  • sbclPackages.clack-static-asset-middleware-test
  • sbclPackages.clack-test
  • sbclPackages.claxy
  • sbclPackages.clerk-test
  • sbclPackages.clgplot-test
  • sbclPackages.clipper
  • sbclPackages.clipper-test
  • sbclPackages.clj
  • sbclPackages.clog
  • sbclPackages.clog-ace
  • sbclPackages.clog-plotly
  • sbclPackages.clog-terminal
  • sbclPackages.clusters-tests
  • sbclPackages.codex
  • sbclPackages.codex-templates
  • sbclPackages.coleslaw-test
  • sbclPackages.colliflower-test
  • sbclPackages.common-doc
  • sbclPackages.common-doc-contrib
  • sbclPackages.common-doc-gnuplot
  • sbclPackages.common-doc-graphviz
  • sbclPackages.common-doc-include
  • sbclPackages.common-doc-plantuml
  • sbclPackages.common-doc-plump
  • sbclPackages.common-doc-plump-test
  • sbclPackages.common-doc-split-paragraphs
  • sbclPackages.common-doc-test
  • sbclPackages.common-doc-tex
  • sbclPackages.common-html
  • sbclPackages.common-html-test
  • sbclPackages.commondoc-markdown
  • sbclPackages.commondoc-markdown-test
  • sbclPackages.configuration_dot_options-and-quri
  • sbclPackages.core-reader_dot_test
  • sbclPackages.cricket_dot_test
  • sbclPackages.cxx-test
  • sbclPackages.datafly-test
  • sbclPackages.dct-test
  • sbclPackages.defclass-std-test
  • sbclPackages.defrest
  • sbclPackages.defrest_dot_test
  • sbclPackages.dexador
  • sbclPackages.dexador-test
  • sbclPackages.dotenv-test
  • sbclPackages.duologue
  • sbclPackages.duologue-readline
  • sbclPackages.duologue-test
  • sbclPackages.eazy-documentation
  • sbclPackages.edit-distance-test
  • sbclPackages.elb-log
  • sbclPackages.elb-log-test
  • sbclPackages.envy-test
  • sbclPackages.event-emitter-test
  • sbclPackages.example-bot
  • sbclPackages.fast-http-test
  • sbclPackages.fast-websocket-test
  • sbclPackages.freesound
  • sbclPackages.fxml
  • sbclPackages.gfxmath_dot_test
  • sbclPackages.github-api-cl
  • sbclPackages.github-gist-api-cl
  • sbclPackages.glacier
  • sbclPackages.glisph-test
  • sbclPackages.hash-table-ext_dot_test
  • sbclPackages.hello-builder
  • sbclPackages.hello-clog
  • sbclPackages.history-tree
  • sbclPackages.house
  • sbclPackages.http-body
  • sbclPackages.http-body-test
  • sbclPackages.ia-hash-table_dot_test
  • sbclPackages.id3v2-test
  • sbclPackages.inner-conditional-test
  • sbclPackages.inquisitor-flexi-test
  • sbclPackages.inquisitor-test
  • sbclPackages.integral-rest
  • sbclPackages.integral-rest-test
  • sbclPackages.integral-test
  • sbclPackages.intel-hex-test
  • sbclPackages.ip-interfaces-test
  • sbclPackages.jingle
  • sbclPackages.jingle_dot_demo
  • sbclPackages.jingle_dot_demo_dot_test
  • sbclPackages.jingle_dot_test
  • sbclPackages.jingoh
  • sbclPackages.jingoh_dot_documentizer_dot_test
  • sbclPackages.jingoh_dot_examiner
  • sbclPackages.jingoh_dot_examiner_dot_test
  • sbclPackages.jingoh_dot_generator_dot_test
  • sbclPackages.jingoh_dot_org_dot_test
  • sbclPackages.jingoh_dot_parallel
  • sbclPackages.jingoh_dot_parallel_dot_test
  • sbclPackages.jingoh_dot_reader
  • sbclPackages.jingoh_dot_reader_dot_test
  • sbclPackages.jingoh_dot_tester
  • sbclPackages.jingoh_dot_tester_dot_test
  • sbclPackages.jonathan-test
  • sbclPackages.json-schema
  • sbclPackages.kebab-test
  • sbclPackages.lack-app-directory
  • sbclPackages.lack-middleware-csrf
  • sbclPackages.lack-middleware-session
  • sbclPackages.lack-request
  • sbclPackages.lack-response
  • sbclPackages.lack-session-store-dbi
  • sbclPackages.lack-session-store-redis
  • sbclPackages.lack-test
  • sbclPackages.lake-test
  • sbclPackages.latter-day-paypal
  • sbclPackages.legion-test
  • sbclPackages.let-over-lambda-test
  • sbclPackages.lisp-pay
  • sbclPackages.lispcord
  • sbclPackages.listopia-bench
  • sbclPackages.lunamech-matrix-api
  • sbclPackages.mailgun
  • sbclPackages.marching-cubes-test
  • sbclPackages.matrix-case_dot_test
  • sbclPackages.mcase_dot_test
  • sbclPackages.millet_dot_test
  • sbclPackages.mito-attachment
  • sbclPackages.mito-test
  • sbclPackages.modest-config-test
  • sbclPackages.mp3-duration-test
  • sbclPackages.multiposter-studio
  • sbclPackages.multiposter-tumblr
  • sbclPackages.multival-plist-test
  • sbclPackages.myway
  • sbclPackages.myway-test
  • sbclPackages.nasdf
  • sbclPackages.nclasses
  • sbclPackages.nfiles
  • sbclPackages.ningle
  • sbclPackages.ningle-test
  • sbclPackages.njson
  • sbclPackages.nodgui
  • sbclPackages.north-dexador
  • sbclPackages.nsort
  • sbclPackages.nsymbols
  • sbclPackages.null-package_dot_test
  • sbclPackages.oclcl-test
  • sbclPackages.openid-key
  • sbclPackages.openid-key-test
  • sbclPackages.oxenfurt
  • sbclPackages.oxenfurt-dexador
  • sbclPackages.pandocl
  • sbclPackages.parenml
  • sbclPackages.parenml-test
  • sbclPackages.path-string-test
  • sbclPackages.perlre
  • sbclPackages.phos
  • sbclPackages.poler-test
  • sbclPackages.ppath-test
  • sbclPackages.proc-parse-test
  • sbclPackages.prometheus_dot_collectors_dot_process_dot_test
  • sbclPackages.prometheus_dot_collectors_dot_sbcl_dot_test
  • sbclPackages.prometheus_dot_exposers_dot_hunchentoot_dot_test
  • sbclPackages.prometheus_dot_formats_dot_text_dot_test
  • sbclPackages.prometheus_dot_pushgateway_dot_test
  • sbclPackages.prometheus_dot_test
  • sbclPackages.prometheus_dot_test_dot_support
  • sbclPackages.prompt-for_dot_test
  • sbclPackages.prompter
  • sbclPackages.prove
  • sbclPackages.prove-test
  • sbclPackages.psychiq-test
  • sbclPackages.quadtree-test
  • sbclPackages.quantile-estimator_dot_test
  • sbclPackages.query-repl_dot_test
  • sbclPackages.quickutil-server
  • sbclPackages.quickutil-utilities-test
  • sbclPackages.quri
  • sbclPackages.quri-test
  • sbclPackages.read-as-string_dot_test
  • sbclPackages.replic
  • sbclPackages.replic-test
  • sbclPackages.resignal-bind_dot_test
  • sbclPackages.restful-test
  • sbclPackages.sanity-clause
  • sbclPackages.scriba
  • sbclPackages.scriba-test
  • sbclPackages.sendgrid
  • sbclPackages.shelly-test
  • sbclPackages.simple-config-test
  • sbclPackages.simple-currency
  • sbclPackages.slack-client
  • sbclPackages.slack-client-test
  • sbclPackages.smart-buffer-test
  • sbclPackages.snooze
  • sbclPackages.snooze-demo
  • sbclPackages.snooze-tests
  • sbclPackages.stepster
  • sbclPackages.str_dot_test
  • sbclPackages.stripe
  • sbclPackages.stripe-against-the-modern-world
  • sbclPackages.structure-ext_dot_as-class_dot_test
  • sbclPackages.structure-ext_dot_left-arrow-accessors_dot_test
  • sbclPackages.structure-ext_dot_make-instance_dot_test
  • sbclPackages.sxql-test
  • sbclPackages.t-clack-handler-fcgi
  • sbclPackages.t-clack-handler-hunchentoot
  • sbclPackages.t-clack-handler-toot
  • sbclPackages.t-clack-handler-wookie
  • sbclPackages.t-lack
  • sbclPackages.t-lack-component
  • sbclPackages.t-lack-middleware-accesslog
  • sbclPackages.t-lack-middleware-auth-basic
  • sbclPackages.t-lack-middleware-backtrace
  • sbclPackages.t-lack-middleware-csrf
  • sbclPackages.t-lack-middleware-mount
  • sbclPackages.t-lack-middleware-session
  • sbclPackages.t-lack-middleware-static
  • sbclPackages.t-lack-request
  • sbclPackages.t-lack-session-store-dbi
  • sbclPackages.t-lack-session-store-redis
  • sbclPackages.t-lack-util
  • sbclPackages.test-gadgets
  • sbclPackages.test-utils
  • sbclPackages.thorn
  • sbclPackages.thorn-doc
  • sbclPackages.thorn-test
  • sbclPackages.thread_dot_comm_dot_rendezvous_dot_test
  • sbclPackages.torrents
  • sbclPackages.torrents-test
  • sbclPackages.translate-client
  • sbclPackages.trestrul_dot_test
  • sbclPackages.trivial-left-pad-test
  • sbclPackages.trivial-string-template-test
  • sbclPackages.trivial-ws-client
  • sbclPackages.trivial-ws-test
  • sbclPackages.ufo-test
  • sbclPackages.vellum-csv-tests
  • sbclPackages.vellum-tests
  • sbclPackages.veq
  • sbclPackages.vertex
  • sbclPackages.vertex-test
  • sbclPackages.vex
  • sbclPackages.vivid-colors
  • sbclPackages.vivid-colors_dot_content
  • sbclPackages.vivid-colors_dot_content_dot_test
  • sbclPackages.vivid-colors_dot_dispatch_dot_test
  • sbclPackages.vivid-colors_dot_queue_dot_test
  • sbclPackages.vivid-colors_dot_shared_dot_test
  • sbclPackages.vivid-colors_dot_stream
  • sbclPackages.vivid-colors_dot_stream_dot_test
  • sbclPackages.vivid-colors_dot_test
  • sbclPackages.vivid-diff
  • sbclPackages.vivid-diff_dot_test
  • sbclPackages.wallstreetflets
  • sbclPackages.webapi
  • sbclPackages.websocket-driver
  • sbclPackages.websocket-driver-client
  • sbclPackages.woo
  • sbclPackages.woo-test
  • sbclPackages.wookie
  • sbclPackages.xsubseq-test
  • sbclPackages.zacl
  • sbclPackages.zenekindarl-test

sbclPackages.nyxt fails because of an heap exhaustion, but nyxt succeeds since it uses wrapLisp with --dynamic-memory-size 2048, It shouldn't really matter since I don't think there's a reason to get Nyxt from sbclPackages rather than the nyxt package since it's an application and not a library.
Also, I'm not sure why it tries to build this many packages is it because of my use of wrapLisp or is it just how nixpkgs-review interacts with changes to lisp-modules/packages.nix?

@Uthar
Copy link
Contributor

Uthar commented May 31, 2023

I also got it to work by registering systems as immutable, I picked this up from upstream:

--- a/pkgs/development/lisp-modules/packages.nix
+++ b/pkgs/development/lisp-modules/packages.nix
@@ -430,13 +430,10 @@ let
       pkgs.gnome.adwaita-icon-theme
     ];
 
-    postConfigure = ''
-      export CL_SOURCE_REGISTRY=$CL_SOURCE_REGISTRY:$(pwd)//
-    '';
-
     buildScript = pkgs.writeText "build-nyxt.lisp" ''
       (load "${super.nyxt.asdfFasl}/asdf.${super.nyxt.faslExt}")
       (asdf:load-system :nyxt/gi-gtk-application)
+      (map () 'asdf:register-immutable-system (asdf:already-loaded-systems))
       (sb-ext:save-lisp-and-die "nyxt" :executable t
                                        #+sb-core-compression :compression
                                        #+sb-core-compression t

I don't know which approach is better though, appending to CL_SOURCE_REGISTRY or registering systems as immutable. Probably we should use whichever and we'll see if issues come up.

@Uthar
Copy link
Contributor

Uthar commented May 31, 2023

Hmm right, so with the flag globally the dependencies would be "shared" between all lisp packages and nyxt. I would vote for increasing the heap globally, in all-packages.nix.

@dariof4
Copy link
Contributor Author

dariof4 commented Jun 1, 2023

I also got it to work by registering systems as immutable, I picked this up from upstream:

--- a/pkgs/development/lisp-modules/packages.nix
+++ b/pkgs/development/lisp-modules/packages.nix
@@ -430,13 +430,10 @@ let
       pkgs.gnome.adwaita-icon-theme
     ];
 
-    postConfigure = ''
-      export CL_SOURCE_REGISTRY=$CL_SOURCE_REGISTRY:$(pwd)//
-    '';
-
     buildScript = pkgs.writeText "build-nyxt.lisp" ''
       (load "${super.nyxt.asdfFasl}/asdf.${super.nyxt.faslExt}")
       (asdf:load-system :nyxt/gi-gtk-application)
+      (map () 'asdf:register-immutable-system (asdf:already-loaded-systems))
       (sb-ext:save-lisp-and-die "nyxt" :executable t
                                        #+sb-core-compression :compression
                                        #+sb-core-compression t

I don't know which approach is better though, appending to CL_SOURCE_REGISTRY or registering systems as immutable. Probably we should use whichever and we'll see if issues come up.

Nice catch! I think using (map () 'asdf:register-immutable-system (asdf:already-loaded-systems)) should be fine since that's what upstream uses.
I did some further research and using asdf:make (which under the hood is just asdf:operate :program-op, it's also what the upstream makefile uses) would also work since it would apply (map () 'asdf:register-immutable-system (asdf:already-loaded-systems)) thanks to the upstream function, but unfortunately I learned that if a system defines :build-operation (like Nyxt) then output translations are ignored for that operation (in this case :program-op) and it tries to place the executable where the .asd file is located, which is in the nix store, so it fails, but if :build-operation is not set then asdf:operate :program-op works normally and respects output translations, which would make it correctly place the executable in /build/source and make the build succeed, so that's a bit annoying :/

Hmm right, so with the flag globally the dependencies would be "shared" between all lisp packages and nyxt. I would vote for increasing the heap globally, in all-packages.nix.

That's how I understood it working, so upping the heap globally would make sense to stop build duplication.

@7c6f434c
Copy link
Member

7c6f434c commented Jun 2, 2023

Does this globally pin quri version in sbcl.pkgs even after we update from quicklisp?

@Uthar
Copy link
Contributor

Uthar commented Jun 2, 2023

Does this globally pin quri version in sbcl.pkgs even after we update from quicklisp?

Yes

@7c6f434c
Copy link
Member

7c6f434c commented Jun 2, 2023

Maybe we should give an attribute name with version included then?

@dariof4
Copy link
Contributor Author

dariof4 commented Jun 2, 2023

Does this globally pin quri version in sbcl.pkgs even after we update from quicklisp?

Since Nyxt uses git submodules to manage their dependencies I went and updated all dependencies that were outdated in quicklisp to the versions pinned by Nyxt, some dependencies may still work if we use their quicklisp version, but to be safe I updated all of the ones that were outdated.
Once quicklisp updates it would be possible to remove a lot of manual updates for the dependencies, but if Nyxt decides to update their submodules in a later version we may run into the same issue of outdated quicklisp dependecies :/
I'm not sure if this is best solution, but I wasn't sure of any other way of doing this.

Maybe we should give an attribute name with version included then?

I'm not sure, since I'm assuming that using newer version of the dependencies should be fine, (but I'm a bit worried about using older versions) so as long as the quicklisp version is at or above what Nyxt uses a pin shouldn't be necessary, so a pin would only be useful as long as it's outdated on quicklisp.

@dariof4
Copy link
Contributor Author

dariof4 commented Jun 2, 2023

Hmm right, so with the flag globally the dependencies would be "shared" between all lisp packages and nyxt. I would vote for increasing the heap globally, in all-packages.nix.

To how much should it be increased?
Doing some testing I found out that Nyxt fails to compile with anything under ~1550MB of dynamic memory size and magicl fails with fewer than ~2300MB of heap size.

@Vonfry
Copy link
Member

Vonfry commented Jun 3, 2023

Since Nyxt uses git submodules to manage their dependencies I went and updated all dependencies that were outdated in quicklisp to the versions pinned by Nyxt, some dependencies may still work if we use their quicklisp version, but to be safe I updated all of the ones that were outdated. Once quicklisp updates it would be possible to remove a lot of manual updates for the dependencies, but if Nyxt decides to update their submodules in a later version we may run into the same issue of outdated quicklisp dependecies :/ I'm not sure if this is best solution, but I wasn't sure of any other way of doing this.

Could we use submodules instead of the dependencies from quicklisp or nix packaged ones? Or, we keep two version of nyxt. One is from quicklisp, and the other is manual packaged.

@bhankas
Copy link
Contributor

bhankas commented Jun 3, 2023

Really good work, @dariof4, it is truly appreciated.

I apologise for being absent on both maintenance front and this PR, but life has been occupying.

Either ways, I've not been using Nyxt for a while now, and don't consider myself qualified enough to maintain nor review. If possible could you please also remove me from the package maintainers? (old name payas, as is present in maintainers, user id is same), TIA!

I will make sure to give the new Nyxt a spin once it reached unstable, quite excited for changes, but my current machine had quite a few troubles in the past so had to stop.

@7c6f434c
Copy link
Member

7c6f434c commented Jun 4, 2023

Silly question: how much RAM to load both Nyxt and magicl? (I know the compiler is hungry and so the question is whether it is «max» or «sum» in practice).

@dariof4
Copy link
Contributor Author

dariof4 commented Jun 4, 2023

Could we use submodules instead of the dependencies from quicklisp or nix packaged ones? Or, we keep two version of nyxt. One is from quicklisp, and the other is manual packaged.

Using submodules instead of quicklisp dependecies is possible, although that would cause some extra building due to not sharing Nyxt's dependecies with other lisp packages.

Really good work, @dariof4, it is truly appreciated.

I apologise for being absent on both maintenance front and this PR, but life has been occupying.

Either ways, I've not been using Nyxt for a while now, and don't consider myself qualified enough to maintain nor review. If possible could you please also remove me from the package maintainers? (old name payas, as is present in maintainers, user id is same), TIA!

I will make sure to give the new Nyxt a spin once it reached unstable, quite excited for changes, but my current machine had quite a few troubles in the past so had to stop.

Okay, I'll remove you from the Nyxt maintainers. Thanks for your previous work maintaining it!

Silly question: how much RAM to load both Nyxt and magicl? (I know the compiler is hungry and so the question is whether it is «max» or «sum» in practice).

I'm not exactly sure how to test that since trying to build Nyxt and magicl simultaneously (nix build .#sbclPackges.{nyxt, magicl}) seems to always fail, I think it's because sb-ext:save-lisp-and-die kills the sbcl process, since this happens with stumpwm as well.

@patrickaldis
Copy link
Contributor

patrickaldis commented Jun 6, 2023

I was just playing around with this PR, and everything runs great, but I realised that I get a lot of warnings in the console about portals:

(WebKitWebProcess:2): Gdk-WARNING **: 10:13:29.316: Failed to read portal settings: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.portal.Settings” on object at path /org/freedesktop/portal/desktop

And noticed that the portal location is pointing to /org/freedeskop/portal/desktop rather than a store location. You should be able to reproduce this by fullscreening a video, the browser should attempt to use portals to inhibit sleep.

Is this an issue on my side that I don't have the right desktop portal installed, or should this point to a store path?

Thanks for all the great work!

@dariof4
Copy link
Contributor Author

dariof4 commented Jun 6, 2023

I was just playing around with this PR, and everything runs great, but I realised that I get a lot of warnings in the console about portals:

(WebKitWebProcess:2): Gdk-WARNING **: 10:13:29.316: Failed to read portal settings: GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface “org.freedesktop.portal.Settings” on object at path /org/freedesktop/portal/desktop

And noticed that the portal location is pointing to /org/freedeskop/portal/desktop rather than a store location. You should be able to reproduce this by fullscreening a video, the browser should attempt to use portals to inhibit sleep.

Is this an issue on my side that I don't have the right desktop portal installed, or should this point to a store path?

Thanks for all the great work!

I unfortunately can't reproduce :(, are you using wayland or x11? And what gpu drivers are you using? (I'm using X11 with the proprietary nvidia drivers on a laptop)
In the meantime I found out about a few other issues:

  1. Copy/Paste doesn't work since (uiop:run-program "PROGRAM" :output :string) tries to create /build for some reason, unless Nyxt is compiled with asdf:make or asdf:operate :program-op 🤷 I also had to add xclip to PATH with makeWrapper.

  2. If you're using nvidia drivers videos don't work unless WEBKIT_DISABLE_COMPOSITING_MODE is set to 1, this seems to be a webkitgtk issues though since ephiphany has the same problem.
    see epiphany: cant play videos #221102 and [WebkitGTK/NVIDIA] Epiphany sometimes doesn't display websites #32580

I updated the branch with the fix for copy/pasting, and I set the --dynamic-space-size flag for sbcl (though this might be better in a separate pr?).

@DieracDelta

This comment was marked as off-topic.

@7c6f434c
Copy link
Member

7c6f434c commented Jun 7, 2023

I think maintainer GitHub id needs to be in the same representation as the other ids in the file

dariof4 added 2 commits June 7, 2023 18:03
Set the `--dynamic-spzce-size` flag for sbcl to add more heap size to
make certain packages be able to compile.
ref: NixOS#235095 (comment)
@dariof4
Copy link
Contributor Author

dariof4 commented Jun 7, 2023

I think maintainer GitHub id needs to be in the same representation as the other ids in the file

Oops, that was a dumb mistake 😅, I fixed it.

@7c6f434c
Copy link
Member

7c6f434c commented Jun 8, 2023

I still think that if we have quicklisp mass-import, then version overrides are better with versions in the attribute names.

(I am not 100% sure what is the proper way to handle Nyxt so that «you can't load Nyxt and magicl at once at all» goes away, but that question is out of scope…)

dariof4 added 4 commits June 10, 2023 19:29
Adds various updated lisp packages that are needed to update nyxt to 3.1.0.
Promter was an internal part of nyxt that was split off as it's own
external library in nyxt 3.0.0.
Package :nyxt/gi-gtk-application instead of :nyxt/gtk-application.
ref: NixOS#217888 (comment)
`WEBKIT_FORCE_SANDBOX=0` is no longer needed since web extension
support has been diabled by default.
ref: atlas-engineer/nyxt#2897

Release notes:
https://nyxt.atlas.engineer/article/release-3.0.0.org
https://github.com/atlas-engineer/nyxt/releases/tag/3.1.0
@dariof4
Copy link
Contributor Author

dariof4 commented Jun 10, 2023

I still think that if we have quicklisp mass-import, then version overrides are better with versions in the attribute names.

(I am not 100% sure what is the proper way to handle Nyxt so that «you can't load Nyxt and magicl at once at all» goes away, but that question is out of scope…)

I went ahead and switched the pr to naming the attributes rather than shadowing them, this required explicitly listing all the lispLibs for Nyxt since if I tried to keep the ones from quicklisp it would prefer quicklisp version of the libraries rather than the updated ones.

@7c6f434c 7c6f434c merged commit 8a9cf23 into NixOS:master Jun 10, 2023
@Janik-Haag Janik-Haag added the 12. first-time contribution This PR is the author's first one; please be gentle! label Jun 13, 2023
@dariof4 dariof4 deleted the nyxt-31 branch June 15, 2023 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update request: nyxt 2.2.4 → 3-pre-release-3
8 participants