Windows installation blocked by Norton/Data Conversion failure

[joeyb1955]

Just thought I would let you know that Norton 360 blocked installation of Brewtarget 40.0.0. It said it would take a couple hours and requires review but never have I come across this issue before with installs of Brewtarget.

[matty0ung]

Sorry to hear you're having problems. I think Windows is gradually getting more strict about code being digitally signed. It's tricky for open source projects because it usually involves buying some expensive certificate. But it looks like at least one company offers special arrangements for open source software (https://about.signpath.io/product/open-source), so I'll get in touch with them.

Hopefully, in the meantime, Norton will let you carry on with the install!

[joeyb1955]

matty0ung,
Update. After giving Norton some time, it did let me continue to work it; it did install.
Next problem is the data conversion. Log appears below along with sql proof the objects complained missing are actually there. Not sure why my 3.0.11 database/schema may not be compliant. Any ideas? Any other information I can provide?

brewtarget.log
[09:27:40.812] (1d74) INFO : Starting Brewtarget v 4.0.0 (app name "brewtarget" ) on "Windows 11 Version 2009" [main.cpp:224]
[09:27:40.812] (1d74) INFO : Built at Wed, Jul 31, 2024 6:03:47 PM on windows for windows with gcc compiler [main.cpp:227]
[09:27:40.812] (1d74) INFO : Log directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:230]
[09:27:40.813] (1d74) INFO : Using Qt runtime v 5.15.14 (compiled against Qt v 5.15.14 ) [main.cpp:231]
[09:27:40.813] (1d74) INFO : Configuration directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:232]
[09:27:40.813] (1d74) INFO : Data directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:233]
[09:27:40.813] (1d74) INFO : void {anonymous}::initResourceDir(QDir&) Application name "brewtarget" [Application.cpp:350]
[09:27:40.813] (1d74) INFO : void {anonymous}::initResourceDir(QDir&) Determined resource directory is "C:/Program Files/Brewtarget-4.0.0/data" [Application.cpp:381]
[09:27:40.813] (1d74) INFO : Resource directory: "C:/Program Files/Brewtarget-4.0.0/data" [main.cpp:234]
[09:27:40.911] (1d74) INFO : bool {anonymous}::ensureDirectoriesExist() Resource directory "C:/Program Files/Brewtarget-4.0.0/data" exists [Application.cpp:149]
[09:27:40.934] (1d74) INFO : Locale: "en_US" (Decimal point: '.' / Thousands separator: ',' ) [Application.cpp:424]
[09:27:40.934] (1d74) INFO : bool Application::initialize() Loading Database... [Application.cpp:441]
[09:27:40.939] (1d74) INFO : bool Database::load() Known DB drivers: ("QIBASE", "QSQLITE", "QMARIADB", "QMYSQL", "QMYSQL3", "QODBC", "QODBC3", "QPSQL", "QPSQL7") [database/Database.cpp:716]
[09:27:40.940] (1d74) INFO : bool Database::impl::loadSQLite(Database&) dbFileName = " C:/Users/jabon/AppData/Local/brewtarget/database.sqlite "
dataDbFileName=" C:/Program Files/Brewtarget-4.0.0/bin/../data/default_db.sqlite " [database/Database.cpp:254]
[09:27:40.963] (1d74) INFO : bool Database::impl::loadSQLite(Database&) SQLite version QVariant(QString, "3.46.0") [database/Database.cpp:303]
[09:27:40.965] (1d74) INFO : bool Database::impl::updateSchema(Database&, bool*) Schema version in DB: 10 , current schema version in code: 12 [database/Database.cpp:400]
[09:27:49.295] (1d74) WARNING : QString::arg: Argument missing: ALTER TABLE settings ADD COLUMN default_content_version, INTEGER [:0]
[09:27:49.896] (1d74) ERROR : bool {anonymous}::executeSqlQueries(BtSqlQuery&, const QVector&) Error executing database upgrade/set-up query "ALTER TABLE hop DROP COLUMN inventory_id" : "error in index hop_inventory_id after drop column: no such column: inventory_id Unable to fetch row" [database/DatabaseSchemaHelper.cpp:90]
[09:27:49.910] (1d74) ERROR : bool Database::impl::updateSchema(Database&, bool*) "Database migration 10->12 failed" [database/Database.cpp:481]

SQLite session:
sqlite> select * from sqlite_schema where name='hop';
table|hop|hop|8|CREATE TABLE hop(
id integer PRIMARY KEY autoincrement,
-- BeerXML properties
name varchar(256) not null DEFAULT '',
alpha real DEFAULT 0.0,
amount real DEFAULT 0.0,
use varchar(32) DEFAULT 'Boil',
time real DEFAULT 0.0,
notes text DEFAULT '',
htype varchar(32) DEFAULT 'Both',
form varchar(32) DEFAULT 'Pellet',
beta real DEFAULT 0.0,
hsi real DEFAULT 0.0,
origin varchar(32),
substitutes text DEFAULT '',
humulene real DEFAULT 0.0,
caryophyllene real DEFAULT 0.0,
cohumulone real DEFAULT 0.0,
myrcene real DEFAULT 0.0,
-- Display stuff
display_unit integer DEFAULT -1,
display_scale integer DEFAULT -1,
-- meta data
deleted boolean DEFAULT 0,
display boolean DEFAULT 1,
folder varchar(256) DEFAULT ''
, inventory_id INTEGER REFERENCES hop_in_inventory(id))
sqlite> select * from sqlite_schema where name='hop_inventory_id';
index|hop_inventory_id|hop|1294|CREATE INDEX 'hop_inventory_id' ON 'hop'('inventory_id')
sqlite> select * from sqlite_schema where name='hop_in_inventory';
table|hop_in_inventory|hop_in_inventory|57|CREATE TABLE "hop_in_inventory" (id INTEGER PRIMARY KEY, amount DOUBLE)
sqlite> select * from sqlite_schema where name ='hop_in_inventory';

[matty0ung]

That's interesting. Your database has at least one index that mine did not. I'll add some logic to drop the index before we drop the column to which it relates. But before I do that, I'm now wondering if there are other indexes etc we need to worry about. Could you run the following and post the results:

SELECT * FROM sqlite_schema WHERE type IN ('index', 'trigger');

[joeyb1955]

Matt Y<

Here you are:

sqlite> SELECT * FROM sqlite_schema WHERE type IN ('index', 'trigger');
trigger|inc_ins_num|instruction_in_recipe|0|CREATE TRIGGER inc_ins_num AFTER INSERT ON instruction_in_recipe
BEGIN
UPDATE instruction_in_recipe SET instruction_number =
(SELECT max(instruction_number) FROM instruction_in_recipe WHERE recipe_id = new.recipe_id) + 1
WHERE rowid = new.rowid;
END
index|sqlite_autoindex_equipment_children_1|equipment_children|48|
index|sqlite_autoindex_fermentable_children_1|fermentable_children|51|
index|sqlite_autoindex_hop_children_1|hop_children|53|
index|sqlite_autoindex_misc_children_1|misc_children|55|
index|sqlite_autoindex_recipe_children_1|recipe_children|628|
index|sqlite_autoindex_style_children_1|style_children|630|
index|sqlite_autoindex_water_children_1|water_children|632|
index|sqlite_autoindex_yeast_children_1|yeast_children|635|
trigger|dec_ins_num|instruction_in_recipe|0|CREATE TRIGGER dec_ins_num AFTER DELETE ON instruction_in_recipe BEGIN UPDATE instruction_in_recipe SET instruction_number = instruction_number - 1 WHERE recipe_id = old.recipe_id AND instruction_number > old.instruction_number; END
index|bt_hop_hop_id|bt_hop|1222|CREATE INDEX 'bt_hop_hop_id' ON 'bt_hop'('hop_id')
index|hop_children_parent_id|hop_children|1224|CREATE INDEX 'hop_children_parent_id' ON 'hop_children'('parent_id')
index|hop_in_recipe_recipe_id|hop_in_recipe|1228|CREATE INDEX 'hop_in_recipe_recipe_id' ON 'hop_in_recipe'('recipe_id')
index|hop_in_recipe_hop_id|hop_in_recipe|1232|CREATE INDEX 'hop_in_recipe_hop_id' ON 'hop_in_recipe'('hop_id')
index|instruction_in_recipe_recipe_id|instruction_in_recipe|1236|CREATE INDEX 'instruction_in_recipe_recipe_id' ON 'instruction_in_recipe'('recipe_id')
index|instruction_in_recipe_instruction_id|instruction_in_recipe|1240|CREATE INDEX 'instruction_in_recipe_instruction_id' ON 'instruction_in_recipe'('instruction_id')
index|equipment_children_parent_id|equipment_children|1249|CREATE INDEX 'equipment_children_parent_id' ON 'equipment_children'('parent_id')
index|misc_inventory_id|misc|1250|CREATE INDEX 'misc_inventory_id' ON 'misc'('inventory_id')
index|misc_children_parent_id|misc_children|1253|CREATE INDEX 'misc_children_parent_id' ON 'misc_children'('parent_id')
index|misc_in_recipe_recipe_id|misc_in_recipe|1255|CREATE INDEX 'misc_in_recipe_recipe_id' ON 'misc_in_recipe'('recipe_id')
index|misc_in_recipe_misc_id|misc_in_recipe|1256|CREATE INDEX 'misc_in_recipe_misc_id' ON 'misc_in_recipe'('misc_id')
index|brewnote_recipe_id|brewnote|1257|CREATE INDEX 'brewnote_recipe_id' ON 'brewnote'('recipe_id')
index|bt_equipment_equipment_id|bt_equipment|1258|CREATE INDEX 'bt_equipment_equipment_id' ON 'bt_equipment'('equipment_id')
index|bt_fermentable_fermentable_id|bt_fermentable|1259|CREATE INDEX 'bt_fermentable_fermentable_id' ON 'bt_fermentable'('fermentable_id')
index|bt_misc_misc_id|bt_misc|1262|CREATE INDEX 'bt_misc_misc_id' ON 'bt_misc'('misc_id')
index|bt_style_style_id|bt_style|1263|CREATE INDEX 'bt_style_style_id' ON 'bt_style'('style_id')
index|bt_water_water_id|bt_water|1264|CREATE INDEX 'bt_water_water_id' ON 'bt_water'('water_id')
index|bt_yeast_yeast_id|bt_yeast|1266|CREATE INDEX 'bt_yeast_yeast_id' ON 'bt_yeast'('yeast_id')
index|fermentable_inventory_id|fermentable|1267|CREATE INDEX 'fermentable_inventory_id' ON 'fermentable'('inventory_id')
index|fermentable_children_parent_id|fermentable_children|1276|CREATE INDEX 'fermentable_children_parent_id' ON 'fermentable_children'('parent_id')
index|fermentable_in_recipe_recipe_id|fermentable_in_recipe|1283|CREATE INDEX 'fermentable_in_recipe_recipe_id' ON 'fermentable_in_recipe'('recipe_id')
index|fermentable_in_recipe_fermentable_id|fermentable_in_recipe|1288|CREATE INDEX 'fermentable_in_recipe_fermentable_id' ON 'fermentable_in_recipe'('fermentable_id')
index|hop_inventory_id|hop|1294|CREATE INDEX 'hop_inventory_id' ON 'hop'('inventory_id')
index|mashstep_mash_id|mashstep|1299|CREATE INDEX 'mashstep_mash_id' ON 'mashstep'('mash_id')
index|recipe_equipment_id|recipe|1302|CREATE INDEX 'recipe_equipment_id' ON 'recipe'('equipment_id')
index|recipe_mash_id|recipe|1304|CREATE INDEX 'recipe_mash_id' ON 'recipe'('mash_id')
index|recipe_style_id|recipe|1305|CREATE INDEX 'recipe_style_id' ON 'recipe'('style_id')
index|recipe_ancestor_id|recipe|1306|CREATE INDEX 'recipe_ancestor_id' ON 'recipe'('ancestor_id')
index|recipe_children_parent_id|recipe_children|1307|CREATE INDEX 'recipe_children_parent_id' ON 'recipe_children'('parent_id')
index|salt_misc_id|salt|1308|CREATE INDEX 'salt_misc_id' ON 'salt'('misc_id')
index|salt_in_recipe_salt_id|salt_in_recipe|1309|CREATE INDEX 'salt_in_recipe_salt_id' ON 'salt_in_recipe'('salt_id')
index|salt_in_recipe_recipe_id|salt_in_recipe|1310|CREATE INDEX 'salt_in_recipe_recipe_id' ON 'salt_in_recipe'('recipe_id')
index|style_children_parent_id|style_children|1311|CREATE INDEX 'style_children_parent_id' ON 'style_children'('parent_id')
index|water_children_parent_id|water_children|1312|CREATE INDEX 'water_children_parent_id' ON 'water_children'('parent_id')
index|water_in_recipe_recipe_id|water_in_recipe|1314|CREATE INDEX 'water_in_recipe_recipe_id' ON 'water_in_recipe'('recipe_id')
index|water_in_recipe_water_id|water_in_recipe|1315|CREATE INDEX 'water_in_recipe_water_id' ON 'water_in_recipe'('water_id')
index|yeast_inventory_id|yeast|1316|CREATE INDEX 'yeast_inventory_id' ON 'yeast'('inventory_id')
index|yeast_children_parent_id|yeast_children|1320|CREATE INDEX 'yeast_children_parent_id' ON 'yeast_children'('parent_id')
index|yeast_in_recipe_recipe_id|yeast_in_recipe|1323|CREATE INDEX 'yeast_in_recipe_recipe_id' ON 'yeast_in_recipe'('recipe_id')
index|yeast_in_recipe_yeast_id|yeast_in_recipe|1324|CREATE INDEX 'yeast_in_recipe_yeast_id' ON 'yeast_in_recipe'('yeast_id')
sqlite>

[matty0ung]

Thanks - that's helpful. If you get a moment, try out https://github.com/Brewtarget/brewtarget/actions/runs/10306286740/artifacts/1791719988 which is a new build that drops the troublesome indexes.

[joeyb1955]

Matt Y,
Ran the install which appeared to run through. Tried to run the 4.0.1 exe with the following result...I did select to install the new data when asked.

[16:55:28.116] (iaw) INFO : Starting Brewtarget v 4.0.1 (app name "brewtarget" ) on "Windows 11 Version 2009" [main.cpp:224]
[16:55:28.116] (iaw) INFO : Built at Thu, Aug 8, 2024 5:33:22 PM on windows for windows with gcc compiler [main.cpp:227]
[16:55:28.116] (iaw) INFO : Log directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:230]
[16:55:28.116] (iaw) INFO : Using Qt runtime v 5.15.14 (compiled against Qt v 5.15.14 ) [main.cpp:231]
[16:55:28.116] (iaw) INFO : Configuration directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:232]
[16:55:28.116] (iaw) INFO : Data directory: "C:/Users/jabon/AppData/Local/brewtarget" [main.cpp:233]
[16:55:28.117] (iaw) INFO : void {anonymous}::initResourceDir(QDir&) Application name "brewtarget" [Application.cpp:350]
[16:55:28.117] (iaw) INFO : void {anonymous}::initResourceDir(QDir&) Determined resource directory is "C:/Program Files/Brewtarget-4.0.1/data" [Application.cpp:381]
[16:55:28.117] (iaw) INFO : Resource directory: "C:/Program Files/Brewtarget-4.0.1/data" [main.cpp:234]
[16:55:28.190] (iaw) INFO : bool {anonymous}::ensureDirectoriesExist() Resource directory "C:/Program Files/Brewtarget-4.0.1/data" exists [Application.cpp:149]
[16:55:28.197] (iaw) INFO : Locale: "en_US" (Decimal point: '.' / Thousands separator: ',' ) [Application.cpp:424]
[16:55:28.198] (iaw) INFO : bool Application::initialize() Loading Database... [Application.cpp:441]
[16:55:28.200] (iaw) INFO : bool Database::load() Known DB drivers: ("QIBASE", "QSQLITE", "QMARIADB", "QMYSQL", "QMYSQL3", "QODBC", "QODBC3", "QPSQL", "QPSQL7") [database/Database.cpp:716]
[16:55:28.200] (iaw) INFO : bool Database::impl::loadSQLite(Database&) dbFileName = " C:/Users/jabon/AppData/Local/brewtarget/database.sqlite "
dataDbFileName=" C:/Program Files/Brewtarget-4.0.1/bin/../data/default_db.sqlite " [database/Database.cpp:254]
[16:55:28.202] (iaw) INFO : bool Database::impl::loadSQLite(Database&) SQLite version QVariant(QString, "3.46.0") [database/Database.cpp:303]
[16:55:28.203] (iaw) INFO : bool Database::impl::updateSchema(Database&, bool*) Schema version in DB: 12 , current schema version in code: 12 [database/Database.cpp:400]
[16:55:28.205] (iaw) INFO : DefaultContentLoader::UpdateResult DefaultContentLoader::updateContentIfNecessary(QSqlDatabase&, QTextStream&) availableContentVersion: 4 , defaultContentAlreadyLoaded: 0 [database/DefaultContentLoader.cpp:97]
[16:55:34.144] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 161 objects from DB table recipe [database/ObjectStore.cpp:1667]
[16:55:34.147] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 36 objects from DB table brewnote [database/ObjectStore.cpp:1667]
[16:55:34.183] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 452 objects from DB table style [database/ObjectStore.cpp:1667]
[16:55:34.215] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 238 objects from DB table equipment [database/ObjectStore.cpp:1667]
[16:55:34.289] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 1166 objects from DB table fermentable [database/ObjectStore.cpp:1667]
[16:55:34.341] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 759 objects from DB table hop [database/ObjectStore.cpp:1667]
[16:55:34.361] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 204 objects from DB table misc [database/ObjectStore.cpp:1667]
[16:55:34.397] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 920 objects from DB table yeast [database/ObjectStore.cpp:1667]
[16:55:34.429] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 26 objects from DB table water [database/ObjectStore.cpp:1667]
[16:55:34.661] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 132 objects from DB table boil [database/ObjectStore.cpp:1667]
[16:55:34.668] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 333 objects from DB table boil_step [database/ObjectStore.cpp:1667]
[16:55:34.671] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 132 objects from DB table fermentation [database/ObjectStore.cpp:1667]
[16:55:34.675] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 167 objects from DB table fermentation_step [database/ObjectStore.cpp:1667]
[16:55:34.681] (iaw) INFO : void ObjectStore::loadAll(Database*) Read 639 objects from DB table instruction [database/ObjectStore.cpp:1667]
[16:55:34.683] (iaw) ERROR : void ObjectStore::impl::wrapAndUnmapAsNeeded(const ObjectStore::TableDefinition&, const ObjectStore::TableField&, QVariant&) Found null value for non-optional Unit when mapping column unit to property unit for fermentable_in_inventory [database/ObjectStore.cpp:938]
[16:55:34.684] (iaw) ERROR : ASSERT: "false" in file ../src/database/ObjectStore.cpp, line 943 [database/ObjectStore.cpp:943]

[matty0ung]

Thanks - that is progress (believe it or not!). The error is saying that, for a particular inventory record of a fermentable, it can't tell whether the amount is a weight or a volume (because now it's allowed to measure some fermentables by volume). When the DB schema got upgraded, it should have set all the fermentable inventory amounts to 'kilograms'. I'm guessing this is either a bug in the upgrade query or an "orphan" inventory row (which we omitted to deal with in the upgrade query).

Would you be able to upload your (pre-upgrade) database.sqlite file? It should be in the same directory with a name like 2024-08-08 18˸51˸27 database.sqlite backup (before upgrade from v9 to v12).

[joeyb1955]

2024-08-08 160711 database.sqlite backup (before upgrade from v10 to v12) - Copy.zip

Requested file attached. I stripped the colons out of the time so I could make a zip.
Thank you for spending the efforts for me. It is appreciated. --joe

[matty0ung]

Thanks - that was super helpful. I have a fix for this and will push out a 4.0.1 release containing it soon (in the next day or two). In the meantime, if you want to get things working without waiting for that, then:

• Restore your pre-upgrade database (ie delete database.sqlite, make a copy of 2024-08-08 160711 database.sqlite backup (before upgrade from v10 to v12), and rename it to database.sqlite)
• Run the following SQL on your pre-upgrade database: DELETE FROM fermentable_in_inventory WHERE id NOT IN (SELECT inventory_id FROM fermentable WHERE inventory_id IS NOT NULL); It should delete a single row (with ID 258) which is a 0 inventory entry for a non-existent fermentable.

The 4.0.1 release will just run this SQL automatically during the upgrade (as well doing the equivalent as on the other inventory tables, though that doesn't affect anything in your particular database).

Please don't hesitate to give us a shout if you hit other problems.

[matty0ung]

Closing now 4.0.1 release is out.