Experiment with a new Protobuf storing format for Inference

[abhinavarora]

Experiment with a new Protobuf message that incorporates the ProgramDesc and Model Parameters for Inference

[kavyasrinet]

My proposal is the following:
In the method save_inference_model, when writing to the content of the file, we can replace the pickle dump with constructing a Protobuf for inference specifically.
We can call this InferenceProgramDesc and the components in this would be :

"program_desc_str": inference_program.desc.serialize_to_string(),
"feed_var_names": feeded_var_names,
"fetch_var_names": fetch_var_names

where program_desc_str is already a ProgramDesc , feed_var_names is a list of variable names that will be fed in and fetch_var_names is also a list of variable names that will be fetched.

Similarly when reading the inference model from the file in load_inference_model in the line , we can replace this with just reading from a protobuf.

This design has the following advantages and disadvantages:

Advantages

1. Easy to implement
2. Does not affect the existing programs that use Fluid.
3. It is robust to changes in the feed_var_names and fetch_var_names variables in the future, or any changes to the signature of the format. The user only needs to update the InferenceProgramDesc file and it should all be in place.

Disadvantages

1. This might look like an ad-hoc strategy for inference, since this is not a standard anywhere else in the codebase and work only for reference.

It'd be great if folks (@Xreki , @kexinzhao ) could talk about their thoughts on this approach.

[Xreki]

• The simplest way is just to add two members in the proto of ProgramDesc, like implemented in Get rid of pickle for fluid inference #7339 . It is reasonable and also common that a ProgramDesc has feed and fetch vars. The advantages and disadvantages are the same as that listed above by @kavyasrinet .

• About design an InferenceDesc which contains ProgramDesc and Parameters. @abhinavarora you mean contains all the values of Parameter? I think the advantages is that it is easy to index all Parameters. I have several questions:

  • How to use load_op and save_op to load and save this Parameters?
  • If designed in this way, is it possible to unify the storing format of train and inference?

  Overal, it needs more designing details.

[abhinavarora]

@Xreki, As per this proposal, The InferenceDesc will contain only the ProgramDesc and the list of feed_variable_names and fetch_variable_names. It will not contain the ParametersWe should make sure that the InferenceDesc and Parameters are stored differently because users might want to save multiple parameter settings for the same Inference program.

The reason to have a separate InferenceDesc instead of adding 2 new fields to the ProgramDesc is that if we need to add more metadata in the future, we do not modify the ProgramDesc every time.