Test the best approach for embeddings

[jfrverdasca]

Description

• Test the best approach to make embeddings as relevant as possible.
• Testing whether we can give the file in chunks to LLM, or whether separating the file into functions (def ...) makes embeddings better and more useful.
• Testing whether adding python docs to each function improves the embeddings and what can be taken from them
• Remove garbage from the code like comments

Requirements

• make a comparison against the same project, with the same issue to compare results

Acceptance Criteria

TO BE DONE

[sIldefonsoRR]

Some of these fine tunes can be done by prompt.

Here are some useful notes:
https://medium.com/@marketing_novita.ai/master-10-top-negative-prompts-examples-and-develop-guide-3a22791aee30#:~:text=Large%20Language%20Models%20(LLMs)%20rely,preventing%20sensitive%20or%20unwanted%20content.

https://medium.com/@marketing_novita.ai/how-to-perform-code-generation-with-llm-models-fe4e10fc5522