Skip to content

Commit

Permalink
Commit translated
Browse files Browse the repository at this point in the history
  • Loading branch information
disenone committed Jan 12, 2024
1 parent cde3490 commit 737e54c
Show file tree
Hide file tree
Showing 2 changed files with 38 additions and 41 deletions.
75 changes: 36 additions & 39 deletions docs/en/py-Python杂谈2-Python312-热更新.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,39 @@
---
layout: post
title: Python Chat 2 - Python 3.12 Hot Update
title: Python Miscellaneous 2 - Python 3.12 Hot Reload
tags:
- dev
- game
- python
- reload
- 热更新
description: How to implement hot reloading in Python 3.12.
description: Recording how to achieve hot reloading in Python 3.12
---

<meta property="og:title" content="Python 杂谈 2 - Python3.12 热更新" />

#Python Misc 2 - Python 3.12 Hot Reload
#Python Miscellaneous 2 - Python 3.12 Hot Update

> Record how to achieve hot reloading in Python 3.12
> Record how to implement hot reloading in Python 3.12
##Hot update

Hot Reload allows updates to be made to a program without the need to restart it. This technology is widely used in the gaming industry. When developers need to fix issues in a game without impacting the players, they often use a technique called hot reload.
Hot Reload is a technology that allows updates to be made to a program without the need for a restart. This technology is widely used in the gaming industry. When developers need to fix issues with games, they often use silent updates, or hot reloads, to avoid impacting players.


##`Python Hot Update`
##Python hot reload

Python itself is a dynamic language, where everything is an object, capable of achieving hot updates. We can roughly divide the objects that need to be updated in Python into two categories: data and functions.
Python itself is a dynamic language, where everything is an object and has the ability to achieve hot updates. We can roughly categorize the objects in Python that need hot updates into two types: data and functions.

Data can be understood as the numerical values or settings in the game, such as the player's level, equipment, and so on. Some data should not be hot-updated (such as the player's current level, the equipment the player has, these data modifications should not be done through hot-updates), some data are the ones we want to hot-update (such as the basic numerical settings of equipment, the basic numerical settings of skills, the text on the UI, and so on).
Data can be understood as the numerical values or settings in the game, such as the player's level, equipment, and so on. Some data should not be updated in real-time. For example, the player's current level, the equipment the player has, should not be modified through real-time updates. Some data is what we want to update in real-time, such as the basic numerical settings of equipment, the basic numerical settings of skills, and the text on the UI, and so on.

Function, can be understood as game logic, this is basically what we want to hotfix, logic errors basically need to be implemented through hot update functions.
Function, can be understood as game logic, which is basically what we want to hotfix, logic errors basically need to be implemented through hot update functions.

Let's take a closer look at what methods can be used to implement hot updates for Python 3.12.
Let's take a closer look at what methods can be used to perform hot reloading on Python 3.12.

## Hotfix

The first method we call Hotfix, which allows the program (both client program and server program) to execute a specific Python code segment to achieve hot updates for data and functions. A simple Hotfix code could look like this:
The first method we call Hotfix, it allows the program (both client and server programs) to execute a specific piece of Python code, achieving hot updates to data and functions. A simple Hotfix code might look like this:


```python
Expand All @@ -51,9 +51,9 @@ def new_fire_func(self, target):
player.Player.fire_func = new_fire_func
```

The above code simply demonstrates the writing method of Hotfix. After the data/function is modified, the program will read the new data/function for execution during subsequent accesses.
The above code simply demonstrates the usage of Hotfix. After the data/function is modified, the program will read the new data/function for subsequent access and execution.

If you are meticulous, you may have a question: What happens if other code references these data and functions that need to be modified?
If you are meticulous, you may have a question: What will happen if other code refers to these data and functions that need to be modified?

```python
# attack.py module
Expand All @@ -65,9 +65,9 @@ def player_attack_by_gun(player, target):
# ...
```

The answer is, the previous Hotfix does not take effect in this situation, the `fire_func` function is like an extra copy in other modules, and the module calls the copy of the function, so the modifications made to the original function do not take effect on the copy.
The answer is that the previous Hotfix does not work for this situation. The function `fire_func` is like an extra copy in another module. The module calls the copy of the function, so the modifications to the original function do not affect the copy.

So need to be careful, generally in the code try to reduce the references to module-level data and function references as much as possible, to avoid the situation where the hotfix does not take effect. If the code is already written this way, the hotfix needs to do more work:
So it's important to note that in general, we should try to minimize module-level data references and function references in the code to avoid situations where Hotfixes don't take effect. If the code is already written this way, extra work will be needed for the Hotfix to work properly:

```python
# hotfix code
Expand All @@ -78,33 +78,31 @@ attack.player_fire = player.Player.fire_func

```

After modifying the data/function core hotfix, make additional modifications to the places where it is referenced. These additional modifications are easily overlooked, so we still recommend avoiding the use of multiple references as much as possible from the code specification.
After modifying the data/function core with a hotfix, make additional modifications to the places where it is referenced. These additional modifications are easily overlooked, so we still recommend avoiding the practice of multiple references as much as possible from the code specification perspective.

In conclusion, Hotfix can meet the basic needs of hot updates, while also presenting the following issues:

**Translate these text into English language:**
In conclusion, Hotfix can meet the basic needs of hot updates, but it also has the following problems:

- If the data/function is explicitly referenced by other modules, additional references to these modules are required. Hotfix
If there's a large amount of data/functions that need to be hotfixed, the hotfix code will become very lengthy, maintenance will become more difficult, and it will also be more prone to errors.
If there is a large amount of data/functions that need to be hotfixed, then the code of the hotfix will become very large, maintenance will become more difficult, and it will also be more prone to errors.

## Reload

The source code for this chapter can be obtained from here: [python_reloader](https://github.com/disenone/python_reloader)
You can obtain the source code for this section from here: [python_reloader](https://github.com/disenone/python_reloader)

What we really want is automatic hot reloading, without the need to write additional hotfixes. We just need to update the code files, and when the program executes a reload function, it will automatically replace the new functions and data. We call this automatic hot reloading function "Reload."
What we really want is automatic hot updates, without the need to write additional hotfixes. Just update the code files, execute a reload function, and the program will automatically replace the new functions and data. We call this automatic hot update feature "Reload."

Python 3.12 provides the importlib.reload function, which can reload modules, but it is a full reload and returns a new module object. For references in other modules, it does not automatically update. In other words, if other modules import the reloaded module, they will still access the old module object. This feature is not much better than our Hotfix, especially considering that it fully reloads the module and does not allow us to control which data should be retained. We want to implement our own reload function to meet these requirements:
Python 3.12 provides the `importlib.reload` function, which can reload modules, but it is a full reload and returns a new module object, so it doesn't automatically update references in other modules. This means that if other modules import the reloaded module, they will still access the old module object. This functionality isn't much better than our Hotfix, especially since it's a full reload of the module and we can't control which data should be retained. We want to implement our own Reload feature that meets these requirements:

- Automatic replacement function, while the reference to the old function remains valid and will execute the content of the new function
- Automatically replace the function, while the reference to the old function remains valid and will execute the content of the new function
- Automatically replace data while controlling partial replacement
- Retain references to the old module, so that the new content can be accessed through the old module.
- Modules that need to be reloaded are controllable.
- Keep the reference to the old module, so that the new content can be accessed through the old module.
- Modules that need reloading are controllable

To accomplish these requirements, we need to make use of the meta_path mechanism inside Python. For a detailed explanation, you can refer to the official documentation [the-meta-path(https://docs.python.org/zh-cn/3/reference/import.html?highlight=meta_path#the-meta-path).
To meet these requirements, we need to leverage the meta_path mechanism within Python. For detailed information, please refer to the official documentation [the-meta-path](https://docs.python.org/zh-cn/3/reference/import.html?highlight=meta_path#the-meta-path)

In `sys.meta_path` we can define our meta-path finder objects. For example, if we name the finder used for reloading as `reload_finder`, `reload_finder` needs to implement a function `find_spec` and return a `spec` object. Upon receiving the `spec` object, Python will sequentially execute `spec.loader.create_module` and `spec.loader.exec_module` to complete the module import.
In `sys.meta_path`, we can define our meta path finder objects, for example, we name the finder used for reloading as `reload_finder`. The `reload_finder` needs to implement a function `find_spec` and return the `spec` object. Once Python gets the `spec` object, it will sequentially execute `spec.loader.create_module` and `spec.loader.exec_module` to complete the module import.

If, during this process, we execute new module code and copy the functions and required data from the new module into the old module, we can achieve the goal of reloading.
If we execute new module code during this process and copy the functions and required data from the new module into the old module, we can achieve the purpose of reloading.

```python linenums="1"
class MetaFinder:
Expand Down Expand Up @@ -150,9 +148,9 @@ class MetaLoader:
module.__loader__ = module.__dict__.pop('__backup_loader__')
```

As mentioned, `find_spec` loads the latest module source code and executes the new module's code within the `__dict__` of the old module. Then we call `ReloadModule` to handle the references and replacements of classes, functions, and data. The purpose of `MetaLoader` is to adapt to the meta_path mechanism and to return the processed module objects to the Python virtual machine.
As mentioned above, `find_spec` loads the latest module source code and executes the new module's code inside the `__dict__` of the old module. Afterwards, we call `ReloadModule` to handle the references and replacements of classes, functions, and data. The purpose of `MetaLoader` is to adapt to the meta_path mechanism and return the processed module objects to the Python virtual machine.

After handling the loading process, let's take a look at the general implementation of `ReloadModule`.
After completing the loading process, let's take a look at the general implementation of `ReloadModule`.

```python linenums="1"
# ...
Expand Down Expand Up @@ -206,24 +204,23 @@ def ReloadDict(self, module, old_dict, new_dict, _reload_all_data=False, _del_fu

```

The `ReloadDict` will differentiate and handle different types of objects.
The `ReloadDict` inside will differentiate and handle different types of objects.

If it's a class, calling `ReloadClass` will return a reference to the old module and update the class members.
If it is a function/method, calling `ReloadFunction` will return a reference to the old module and update the internal data of the function.
If it's a class, calling `ReloadClass` will return the reference to the old module and update the class members.
If it is a function/method, calling `ReloadFunction` will return a reference to the old module and update the internal function data.
If it's data and needs to be preserved, it will roll back `new_dict[attr_name] = old_attr`
- The rest should keep the new reference.
- The rest should remain with the new citation.
- Remove functions that do not exist in the new module

The specific code for `ReloadClass` and `ReloadFunction` will not be further analyzed here. If you are interested, you can directly check the [source code(https://github.com/disenone/python_reloader)。
The specific code for `ReloadClass` and `ReloadFunction` will not be further analyzed here. If you are interested, you can directly refer to the [source code](https://github.com/disenone/python_reloader)

The whole Reload process can be summarized as "old wine in a new bottle". To keep the module/module function/module class/module data valid, we need to retain references to these original objects (shells) and update their specific data internally, for example, for functions, update `__code__`, `__dict__`, etc. When the function is executed, it will then execute the new code instead.
The entire process of Reload can be summarized as "putting new wine in old bottles." In order to maintain the effectiveness of modules, module functions, module classes, and module data, we need to retain references to these original objects (shells) and then update their specific data internally. For example, for functions, we update `__code__`, `__dict__`, and other data. When the function is executed, it will then execute the new code.

##Summary

This article provides a detailed introduction to two hot update methods of Python3, each with its corresponding application scenarios, hoping to be helpful to you. If you have any questions, please feel free to communicate at any time.
This article provides a detailed introduction to two hot update methods of Python3, each with its corresponding use cases, hoping to be helpful to you. Feel free to reach out if you have any questions.

--8<-- "footer_en.md"



> This post is translated using ChatGPT, please [**feedback**](https://github.com/disenone/wiki/issues/new) if any omissions.
4 changes: 2 additions & 2 deletions tools/processed_dict.txt
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@
"mtime": 1703786987.3951695
},
"py-Python杂谈2-Python312-热更新.md": {
"git_ref": "46c6b2c",
"mtime": 1705046660.736937
"git_ref": "cde3490",
"mtime": 1705047409.491685
}
}

0 comments on commit 737e54c

Please sign in to comment.