Mint removes Python modules if modules are loaded dynamically

[jkempf42]

If you include code like this:

                module = importlib.import_module(module_name)

which dynamically loads the module. I suspect mint is currently just following the imports down from the top level function, which won't catch dynamically loaded modules.

[kcq]

@jkempf42 Is this a server app or a CLI app? This python app is a good candidate for additional probe parameters. The goal of those probes would be to trigger the flow where that module is loaded dynamically. There are several probe types. The HTTP/network probes (configured using the --http-probe-* flags) interact with the server applications. The exec probes (configured using the --exec and --exec-file flags) allow you to run extra commands in the ephemeral container when the application is running.

[jkempf42]

It is an Azure Functions function app, so a server app. jak

…

On Thu, Feb 13, 2025 at 10:59 AM Kyle Quest ***@***.***> wrote: @jkempf42 <https://github.com/jkempf42> Is this a server app or a CLI app? This python app is a good candidate for additional probe parameters. The goal of those probes would be to trigger the flow where that module is loaded dynamically. There are several probe types. The HTTP/network probes (configured using the --http-probe-* flags) interact with the server applications. The exec probes (configured using the --exec and --exec-file flags) allow you to run extra commands in the ephemeral container when the application is running. — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GVFQWEYGUWHWF22HAD2PTTRBAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJXGQ3TIMRSHE> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) <#752 (comment)> @jkempf42 <https://github.com/jkempf42> Is this a server app or a CLI app? This python app is a good candidate for additional probe parameters. The goal of those probes would be to trigger the flow where that module is loaded dynamically. There are several probe types. The HTTP/network probes (configured using the --http-probe-* flags) interact with the server applications. The exec probes (configured using the --exec and --exec-file flags) allow you to run extra commands in the ephemeral container when the application is running. — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GVFQWEYGUWHWF22HAD2PTTRBAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNJXGQ3TIMRSHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kcq]

@jkempf42 this sounds promising! Is there a simple example of for a containerized Azure Function you'd recommend for a repro? Maybe an open source project or just sample code?

[jkempf42]

I'll send sample code or a link to a repo with it in. It will use the Microsoft Azure Functions containerized function base image and a simple example of a function that dynamically loads a module. jak

…

On Sat, Feb 15, 2025 at 11:22 AM Kyle Quest ***@***.***> wrote: @jkempf42 <https://github.com/jkempf42> this sounds promising! Is there a simple example of for a containerized Azure Function you'd recommend for a repro? Maybe an open source project or just sample code? — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GWUUDWSQNL4SJ3D3PT2P6HX5AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGA2TKNRUG4> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) <#752 (comment)> @jkempf42 <https://github.com/jkempf42> this sounds promising! Is there a simple example of for a containerized Azure Function you'd recommend for a repro? Maybe an open source project or just sample code? — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GWUUDWSQNL4SJ3D3PT2P6HX5AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGA2TKNRUG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kcq]

@jkempf42 thank you! looking forward to exploring those and confirming the flags to use with those containers.

[jkempf42]

@kcq, please see the public repo https://github.com/jkempf42/mint-bug for an example. In the process of coming up with the example, I discovered another bug where mint removes some code used by the Azure Functions runtime to return values. The code appears to be a C# module. The README.md file contains instructions for both bugs, and there are two bash scripts that you can execute against the container that illustrate each bug separately. Enjoy! jak

…

On Sun, Feb 16, 2025 at 10:18 AM Kyle Quest ***@***.***> wrote: @jkempf42 <https://github.com/jkempf42> thank you! looking forward to exploring those and confirming the flags to use with those containers. — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GQFRFW5ZRAAEB6I2JD2QDI55AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGU2TKMZXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) <#752 (comment)> @jkempf42 <https://github.com/jkempf42> thank you! looking forward to exploring those and confirming the flags to use with those containers. — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GQFRFW5ZRAAEB6I2JD2QDI55AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGU2TKMZXGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kcq]

@jkempf42 try this command: mint build --expose 80 --http-probe-cmd '/api/mint_bug_example?name=Darth_Vader' --http-probe-cmd '/api/mint_bug_example?name=Chauncy_Laubach' mint-bug:latest

[jkempf42]

Well, that worked but it is hardly a solution. In the original bug, the dynamic loading of a Python module was triggered by a header parameter, not a query parameter. And is a developer supposed to go through all the possible values for a query or header parameter? That's not scalable. jak

…

On Sun, Feb 16, 2025 at 11:34 AM Kyle Quest ***@***.***> wrote: @jkempf42 <https://github.com/jkempf42> try this command: mint build --expose 80 --http-probe-cmd '/api/mint_bug_example?name=Darth_Vader' --http-probe-cmd '/api/mint_bug_example?name=Chauncy_Laubach' mint-bug:latest — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GREVKU7WIHMVGGWA6T2QDR43AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGU4DGMBUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) <#752 (comment)> @jkempf42 <https://github.com/jkempf42> try this command: mint build --expose 80 --http-probe-cmd '/api/mint_bug_example?name=Darth_Vader' --http-probe-cmd '/api/mint_bug_example?name=Chauncy_Laubach' mint-bug:latest — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GREVKU7WIHMVGGWA6T2QDR43AVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGU4DGMBUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[kcq]

@jkempf42 the --http-probe-cmd-file flag gives you more control over the HTTP requests including the headers. Here's a sample command definition file for that flag has a custom request header defined:

{
  "commands":
  [
   {
     "protocol": "http",
     "method": "POST",
     "resource": "/submit2",
     "body": "key=value"
   },
   {
     "protocol": "http",
     "method": "POST",
     "resource": "/submit3",
     "body_file": "mydata.json",
     "headers": ["Content-Type: application/json"]
   }
  ]
}

You don't need to go through all possible value, but you do need to be aware of special cases. Usually there are only a few cases like that.

Good to note that with servers/services that have swagger/OpenAPI API specs you can point to those (with the --http-probe-apispec or --http-probe-apispec-file flags) and that reduces the number of custom HTTP probe commands you need to specify explicitly.

There's also an automated way to generate the necessary probes, but it goes beyond the capabilities of the toolkit because additional context is needed. This autonomous probing capability is available in the AppDroid service we have (available in closed beta at this point in time).

[jkempf42]

Good to know about the OpenAPI specs. I have OpenAPI 3.1 specs for all my APIs. I'll give that a try. Thanx! jak

…

On Sun, Feb 16, 2025 at 2:05 PM Kyle Quest ***@***.***> wrote: @jkempf42 <https://github.com/jkempf42> the --http-probe-cmd-file flag gives you more control over the HTTP requests including the headers. Here's a sample command definition file for that flag has a custom request header defined: { "commands": [ { "protocol": "http", "method": "POST", "resource": "/submit2", "body": "key=value" }, { "protocol": "http", "method": "POST", "resource": "/submit3", "body_file": "mydata.json", "headers": ["Content-Type: application/json"] } ] } You don't need to go through all possible value, but you do need to be aware of special cases. Usually there are only a few cases like that. Good to note that with servers/services that have swagger/OpenAPI API specs you can point to those (with the --http-probe-apispec or --http-probe-apispec-file flags) and that reduces the number of custom HTTP probe commands you need to specify explicitly. There's also an automated way to generate the necessary probes, but it goes beyond the capabilities of the toolkit because additional context is needed. This autonomous probing capability is available in the AppDroid service we have (available in closed beta at this point in time). — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GVQAPRM4KLUYWIUKV32QEDTZAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGYZTSNRQGI> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) <#752 (comment)> @jkempf42 <https://github.com/jkempf42> the --http-probe-cmd-file flag gives you more control over the HTTP requests including the headers. Here's a sample command definition file for that flag has a custom request header defined: { "commands": [ { "protocol": "http", "method": "POST", "resource": "/submit2", "body": "key=value" }, { "protocol": "http", "method": "POST", "resource": "/submit3", "body_file": "mydata.json", "headers": ["Content-Type: application/json"] } ] } You don't need to go through all possible value, but you do need to be aware of special cases. Usually there are only a few cases like that. Good to note that with servers/services that have swagger/OpenAPI API specs you can point to those (with the --http-probe-apispec or --http-probe-apispec-file flags) and that reduces the number of custom HTTP probe commands you need to specify explicitly. There's also an automated way to generate the necessary probes, but it goes beyond the capabilities of the toolkit because additional context is needed. This autonomous probing capability is available in the AppDroid service we have (available in closed beta at this point in time). — Reply to this email directly, view it on GitHub <#752 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADE62GVQAPRM4KLUYWIUKV32QEDTZAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGYZTSNRQGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jkempf42]

@kcq, I tried my production image using --http-probe-apispec-file and the same problem happened with the dynamically loaded module. The way it works is that a header parameter contains the name of a Python class, which is then reformatted into the module name for the class and the module is loaded using importlib. I provided mint with OpenAPI 3.1 definitions for all the API requests; however, mint still removed the module which was dynamically loaded. jak

…

On Sun, Feb 16, 2025 at 2:24 PM James Kempf ***@***.***> wrote: Good to know about the OpenAPI specs. I have OpenAPI 3.1 specs for all my APIs. I'll give that a try. Thanx! jak On Sun, Feb 16, 2025 at 2:05 PM Kyle Quest ***@***.***> wrote: > @jkempf42 <https://github.com/jkempf42> the --http-probe-cmd-file flag > gives you more control over the HTTP requests including the headers. Here's > a sample command definition file for that flag has a custom request header > defined: > > { > "commands": > [ > { > "protocol": "http", > "method": "POST", > "resource": "/submit2", > "body": "key=value" > }, > { > "protocol": "http", > "method": "POST", > "resource": "/submit3", > "body_file": "mydata.json", > "headers": ["Content-Type: application/json"] > } > ] > } > > You don't need to go through all possible value, but you do need to be > aware of special cases. Usually there are only a few cases like that. > > Good to note that with servers/services that have swagger/OpenAPI API > specs you can point to those (with the --http-probe-apispec or > --http-probe-apispec-file flags) and that reduces the number of custom > HTTP probe commands you need to specify explicitly. > > There's also an automated way to generate the necessary probes, but it > goes beyond the capabilities of the toolkit because additional context is > needed. This autonomous probing capability is available in the AppDroid > service we have (available in closed beta at this point in time). > > — > Reply to this email directly, view it on GitHub > <#752 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADE62GVQAPRM4KLUYWIUKV32QEDTZAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGYZTSNRQGI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> > [image: kcq]*kcq* left a comment (slimtoolkit/slim#752) > <#752 (comment)> > > @jkempf42 <https://github.com/jkempf42> the --http-probe-cmd-file flag > gives you more control over the HTTP requests including the headers. Here's > a sample command definition file for that flag has a custom request header > defined: > > { > "commands": > [ > { > "protocol": "http", > "method": "POST", > "resource": "/submit2", > "body": "key=value" > }, > { > "protocol": "http", > "method": "POST", > "resource": "/submit3", > "body_file": "mydata.json", > "headers": ["Content-Type: application/json"] > } > ] > } > > You don't need to go through all possible value, but you do need to be > aware of special cases. Usually there are only a few cases like that. > > Good to note that with servers/services that have swagger/OpenAPI API > specs you can point to those (with the --http-probe-apispec or > --http-probe-apispec-file flags) and that reduces the number of custom > HTTP probe commands you need to specify explicitly. > > There's also an automated way to generate the necessary probes, but it > goes beyond the capabilities of the toolkit because additional context is > needed. This autonomous probing capability is available in the AppDroid > service we have (available in closed beta at this point in time). > > — > Reply to this email directly, view it on GitHub > <#752 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ADE62GVQAPRM4KLUYWIUKV32QEDTZAVCNFSM6AAAAABXCXF3SSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRRGYZTSNRQGI> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >