Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Post Processing passes wrong name #7190

Closed
tomvandeneede opened this issue Oct 28, 2021 · 79 comments
Closed

Post Processing passes wrong name #7190

tomvandeneede opened this issue Oct 28, 2021 · 79 comments

Comments

@tomvandeneede
Copy link

Version

Version 2.4.0-beta1+x64

Operating system type + version

MacOS Monterey

3D printer brand / version + firmware version (if known)

Prusa MK3S

Behavior

Running P2PP post processing script for Palette3 from Mosaic

Script gets the file name
/var/folders/fl/3n88wwgj01j2nbvs26tzs8lr0000gn/T/.43983.gcode.pp

in the folder I find the following file
2712 -rw-r--r-- 1 tomvandeneede staff 1388386 Oct 28 20:40 .43983.gcode

Seems that the .pp extension is missing in the name of the output file.
This still worked ok in Alpha 3

No new feature request

@tomvandeneede
Copy link
Author

image
Adding the screenshot for additional info

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 29, 2021

Please check the alpha1 change log for post-processing script changes, namely

G-code post-processor framework has been extended: #6042
New environment variable SLIC3R_PP_HOST contains one of "File", "PrusaLink", "Repetier", "SL1Host", "OctoPrint", "FlashAir", "Duet", "AstroBox" ...
New environment variable SLIC3R_PP_OUTPUT_NAME contains the name of the G-code file including path (for SLIC3R_PP_HOST == "File") or a name that will be given to the file after it is upload to the host (PrusaLink, Octoprint ...)
The post-processing script may suggest a new output file name (likely based on SLIC3R_PP_OUTPUT_NAME) by saving it as a single line into a new "output name" temp file. The "output name" file name is created by suffixing the input G-code file name with ".output_name".

The generated G-code may be locked for reading due to the G-code viewer, thus P2PP shall open the G-code for reading only or to make a copy of it first.

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 29, 2021

MacOS Monterey

Didn't you update your system? Maybe the .pp extension is trimmed by the operating system?

Does alpha3 work for you on the same system?

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 29, 2021

Please provide us with your 3MF and possibly a hint on how to install your post-processor.

BTW are you the post-processor author?

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 29, 2021

a 3MF file that calls your PP please

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 29, 2021

sorry nothing attached. You may want to zip the 3mf or change its extension

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Oct 30, 2021

PrusaSlicer expects that the post processing script does its thing on the G-code until its process finishes. If you do start another process to work with the G-code "offline", than PrusaSlicer has no way of knowing it. Do you do that?

@tomvandeneede
Copy link
Author

tomvandeneede commented Oct 30, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 1, 2021

Me: a 3MF file that calls your PP please
You: this one should use the pp
Me: sorry nothing attached. You may want to zip the 3mf or change its extension?

Would you please provide your 3MF? It contains the launch command for your post-processing script. Thanks a bunch.

@tomvandeneede
Copy link
Author

I have tried to reproduce on several systems and the behaviour is strange:

Palette 3 processing script needs a special output file format names .mcfx (not a plain gcode file, binary format)

when putting an output pattern .mxfc, some people get the desired.mcfx, others get the correctly processed file but with with a filename with a .gcode suffix.

The processing script does not handle any of the environment variables at this point (planning to in a later release).

So the filename seems to be passed ok, processing happens ok, but the final output file gets the wrong filename.
That is where I landed with my testing to date.

Should I raise a new issue and close this one as they are not really the same. The oridginal issue was solely due to async processing.

@magnificu
Copy link

On Ubuntu 20.04 the post processing script runs in /tmp/ folder instead on the folder I saved the *.gcode file
Is there any way to get the global path of the g-code (on the folder I save it) and pass it to the post script?

@foreachthing
Copy link

Inside the Python PostProcessing Script you can use:
(no need to pass it manually, PS does it with this env-var)

from os import getenv
## get envvar from PrusaSlicer
env_slicer_pp_output_name = str(getenv('SLIC3R_PP_OUTPUT_NAME'))

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 6, 2021 via email

@magnificu
Copy link

After "export to SD Card", in the /tmp/ folder I have two files:
.10219.gcode (original gcode)
.10219.gcode.pp (post processed gcode)

In the folder I "export to SD Card" the gcode (from gui) is saved only the "original gcode"

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 8, 2021

@tomvandeneede Would you please provide a single 3MF please? This is the fifth time I am asking you to do me the favor of providing me with a single 3MF to save my time invested into troubleshooting your problem. Thank you.

I think the issue is that your "script" is a Windows GUI application, not a Windows command line application, thus PrusaSlicer likely does not wait for your process to finish. I think this is a feature of Windows system. Please

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 8, 2021

This is why we provide two PrusaSlicer binaries on Windows: prusa-slicer.exe and prusa-slicer-console.exe. In fact the two only differ in a single bit in the application EXE header. We pack the complete application into a DLL, which is then launched by these two wrappers differing in EXE header only, one indicating a GUI application and the other indicating a command line application.

The source code to generate these wrappers is here:
https://github.com/prusa3d/PrusaSlicer/blob/master/src/CMakeLists.txt
https://github.com/prusa3d/PrusaSlicer/blob/master/src/PrusaSlicer_app_msvc.cpp

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 8, 2021 via email

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 8, 2021 via email

@tomvandeneede
Copy link
Author

sorry no time to further help on this. I just relayed information received from other users. The whole postprocessing system has gotten way more messy than it should be ... I hope somebody does get the benefit of these changes... I will close this issues not resolved but don't know what to do if no responses are read any only 3MF files are asked....

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 8, 2021

I will close this issues not resolved but don't know what to do if no responses are read any only 3MF files are asked....

@tomvandeneede I am sorry that I disappointed you, however my schedule is tight.

With all respect, it may be I overlooked your 3MF attached to this issue, but I don't see it. Please paste the hyperlink to the 3MF that you managed to attach into this issue. A 3MF helps us to replicate your workflow exactly, it helps us a lot.

If you did not manage to attach a 3MF to this issue, you may pass it through drop box for example, there are multiple ways to do it.

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 8, 2021

All post processing goes just fine, only the filename at the end seems to differ from the filename provided in the templated form in the output options.

So in the pp script I am getting a .pp file name and I store my resulting file in that very same place. With some users the output filename is .gcode while they have .mcfx defined in their settings
There is no apparent reason for this. I haven’t been able to reproduce the issue but have had several reports from script users

Is this remaining issue platform specific? OSX only?

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 8, 2021 via email

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 9, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 10, 2021

@tomvandeneede

If I understand it correctly, the issue is OSX specific only and it is connected with the system default file output dialog that PrusaSlicer opens when exporting a G-code, which forces a wrong (default) extension? BTW this is not a new issue, IMHO this is how PrusaSlicer on OSX always worked. We need to list out the allowed extensions in the file dialog API explicitly to work on OSX.

@magnificu
Copy link

I confirm that in PrusaSlicer 2.4 Beta2 the issue on Kubuntu is not fixed.
I provided already the 3MF file and the python script.

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 29, 2021 via email

@sg000
Copy link

sg000 commented Nov 29, 2021

@bubnikv

PrusaSlicer (and Slic3r) always expected that the file will be modified in place, thus the resulting G-code will be stored under the input file name. Your script modifies the output file name. From input file name .27432.gcode.pp you produce .27432.gcode.pp.gcode, while you should have produced .27432.gcode.pp

I've never seen a .gcode.pp or gcode.pp.gcode file in my folder. I've always gotten .gcode in the end (plus my copied pre-modified bumpfan.bk file). I'll look through release notes, but I don't recall seeing any compat busters for post-processing output filename changes. Did I miss something?

BTW, not OSX here, I'm in native Windows 10.

@sg000
Copy link

sg000 commented Nov 29, 2021

I found why I have a problem. See the filename argument sent from two different PrusaSlicer vserions: v2.4 above and v2.3 below. 2 problems.

  1. the filename sent to post-processing in 2.4 is appended by .pp (why?) AND isn't the final filename the user gave the file to save to.
  2. the folder for the file sent to post-processing in 2.4 is a temporary folder, not the final save folder.

Again, why? Doesn't post-processing mean AFTER everything else is done?

The whole purpose of my script to make a copy is to allow me to compare the old and new to make sure the script did what I wanted it to.

Screenshot 2021-11-29 071321
:

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

From PrusaSlicer 2.4.0-alpha1:

G-code post-processor is newly executed before the G-code is copied to the target location. Indeed, running a post-processing script on SD card is not a good idea. Please note that the G-code viewer still visualizes G-code before post-processing
#6000 thanks @xorza.

G-code post-processor framework has been extended: #6042
New environment variable SLIC3R_PP_HOST contains one of "File", "PrusaLink", "Repetier", "SL1Host", "OctoPrint", "FlashAir", "Duet", "AstroBox" ...
New environment variable SLIC3R_PP_OUTPUT_NAME contains the name of the G-code file including path (for SLIC3R_PP_HOST == "File") or a name that will be given to the file after it is upload to the host (PrusaLink, Octoprint ...)
The post-processing script may suggest a new output file name (likely based on SLIC3R_PP_OUTPUT_NAME) by saving it as a single line into a new "output name" temp file. The "output name" file name is created by suffixing the input G-code file name with ".output_name".

@foreachthing
Copy link

foreachthing commented Nov 29, 2021

Again, why? Doesn't post-processing mean AFTER everything else is done?

Because someone thought it'd be better (and faster; yes it is faster ... BUT!!!) to do "post-processing" before the file gets copied to his/her sd-card.
Just because of this one commit, the whole post-processing got fubared, if you ask me. Prusa then programmed days on a working solution and still people report problems. Just undo that commit #6000 and we're done here!
duckandruninzickzackandtakecover

@bubnikv please don't take it personally!!!! It's just frustration. My script works now (THANKS to you PS-guys for helping!). I just hate to see other people struggling with the same thing.

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@foreachthing I don't understand why you are so vocal about it. Why don't you just update your scripts? I should send you guys an invoice for the hours spent on it.

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@tomvandeneede

I did some more experiments on OSX. It seems as if the problem was, that if one does not enter an extension to the file dialog input field, the default file extension is applied. The default file extension is the first one in the template list. If one adds an extension that is contained in the list (and we are making sure programmatically that the file extension IS in the list if it was provided as part of the output file template), then everything works and the file is saved using the non-default extension.

Now what is weird is that one would expect if the save file dialog is opened with a path that contains a non-default file extension which is still in the template list (we are making sure it is the case), then the file dialog would rememeber the extension. But that is not the case and the extension is thrown away by the file dialog and substituted with the default one.

We will try to redo the routine which handles the file extension template list to put the one that is contained in the output file name template parameter to the first place. That should solve this particular issue for good.

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@tomvandeneede We can whitelist your extension, but I am starting to be positive (and I am also COVID positive) about the solution.

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 29, 2021 via email

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@tomvandeneede I think the OSX file output dialog issue should be fixed with e7cc12b
Fix of output file dialog extension template for OSX:
When opening a "Save" file dialog with a default output file name,
OSX file dialog strips the provided extension and replaces it
with the default extension. This causes issues with a custom
file output template, where a non-default file extension is specified.

This commit changes the function to generate file dialog templates
so that if a default extension is provided, it is emitted
as first into the extension template.

I need to pipe it through the build server and test. However on Windows the code seems to be doing what it should.

bubnikv added a commit that referenced this issue Nov 29, 2021
When opening a "Save" file dialog with a default output file name,
OSX file dialog strips the provided extension and replaces it
with the default extension. This causes issues with a custom
file output template, where a non-default file extension is specified.

This commit changes the function to generate file dialog templates
so that if a default extension is provided, it is emitted
as first into the extension template.

Fixes Post Processing passes wrong name #7190
@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@tomvandeneede Thus for the fix to work, you will need to specify your custom file extension in output file template.

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@foreachthing The post processing script handling is now technically sound and well defined, better than before if you ask me. We need to document it well.

We may add a check for existence of the input / output G-code file and emit an explanation, why PrusaSlicer complains that the file is missing.

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 29, 2021

@tomvandeneede
Would you please test this OSX build whether it solves your issue? Please note that you need to set the output file name template correctly for the extension substitution to work correctly.

https://prusaslicer:[email protected]/files/PrusaSlicer-2.4.0-beta2+6-universal-asan-ge7cc12b2c-202111291519.dmg

@magnificu
Copy link

magnificu commented Nov 29, 2021

@bubnikv

Test data received, however your python "script" requires quite some number of large python dependencies to be installed.

Simple question. The script's first parameter is the script file name. Do you save your results into the same file? This is expected by PrusaSlicer.

Yes. The first parameter expected by the script is the global path including the gcode file name.
When I run it in linux from a terminal it works as expected, but not from PrusaSlicer

I modified the python script to print the file name he gets from PrusaSlicer and this is the result:
/tmp/.2068.gcode.pp

the expected file name was this:
/media/user/Data1/3D_Models/magni3D_v005/gcode_prusa/Belt_Tensioner_02_v005_0.2mm_PLA__15m.gcode

@foreachthing
Copy link

@magnificu
This should give you the correct output filename: env_slicer_pp_output_name = str(getenv('SLIC3R_PP_OUTPUT_NAME')).
The passed parameter is the temp filename.

@magnificu
Copy link

@foreachthing This should give you the correct output filename: env_slicer_pp_output_name = str(getenv('SLIC3R_PP_OUTPUT_NAME')). The passed parameter is the temp filename.

How can I pass this to python?

@tomvandeneede
Copy link
Author

tomvandeneede commented Nov 30, 2021 via email

@sg000
Copy link

sg000 commented Nov 30, 2021

From PrusaSlicer 2.4.0-alpha1:

G-code post-processor is newly executed before the G-code is copied to the target location. Indeed, running a post-processing script on SD card is not a good idea. Please note that the G-code viewer still visualizes G-code before post-processing
#6000 thanks @xorza.
@bubnikv Thanks. I found it myself (and should have looked in the first place, my bad) when I got to the office, and read up on things. Couldn't reply until now, though.

@xorza I can change my script I think easy enough, but for the record, I'll echo what others have said about post-processing being actually post. Pre and Post? Toggle? The env vars are a really nice addition, but doesn't fix the root confusion. There should be a note in the GUI to advise, especially of a change. Not a tooltip either, that's too subtle for this type of thing imoho.

Do appreciate all the hard work all of you do. I've been where all you contributors are before. Truly, Thank You.

--sg000

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 30, 2021

@sg000

I have added the following error dialog in case the G-code is not being edited in place by the post-processing script:

image

This should make the situation a bit better.
f51162f

@bubnikv
Copy link
Collaborator

bubnikv commented Nov 30, 2021

BTW https://manual.slic3r.org/advanced/post-processing

Each script will be passed the absolute path of the G-code file that Slic3r generates. The script will need to modify it in-place; alternatively it can output the new G-code to a temporary file and then overwrite the original one with it.

It was always stated in the manual, that the post-processor shall modify the G-code in place.

@magnificu
Copy link

magnificu commented Nov 30, 2021

@foreachthing https://github.com/foreachthing This should give you the correct output filename: env_slicer_pp_output_name = str(getenv('SLIC3R_PP_OUTPUT_NAME')). The passed parameter is the temp filename. How can I pass this to python? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7190 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK6CHJLRAVINDIWTAYC6KZTUOR537ANCNFSM5G5SN5SQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

Thank you for your feedback.
The variable seems to be empty...

@magnificu
Copy link

How can I pass this to python?

https://github.com/foreachthing/Slic3rPostProcessing/blob/a47d64c8b83459cf3bd2906a3e81172c07328f34/SPP-Python/Slic3rPostProcessor.py#L212

Thank you.
I get the path, but for some reason the script is executed before having the file saved in the location

@foreachthing
Copy link

Thank you. I get the path, but for some reason the script is executed before having the file saved in the location

That's the "new and improofed" post processing. Better get used to it. :-\

@bubnikv
Copy link
Collaborator

bubnikv commented Dec 4, 2021

I think we can close this issue now.

The original OSX issue was fixed with e7cc12b and @tomvandeneede was passed a build for testing, while no response was received.

The post processing script was documented with
https://help.prusa3d.com/en/article/post-processing-scripts_283913
there is now a comment at the post processing script parameter page which also points to the help page above. Also if one renames the temp G-code with his post processing script, the error is reported f51162f.

Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants