Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync : ggml (backend v2) #3912

Merged
merged 19 commits into from
Nov 13, 2023
Merged

sync : ggml (backend v2) #3912

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
aa7a2c4
sync : ggml (backend v2) (wip)
ggerganov Nov 2, 2023
e819070
sync : migrate examples and llama.cpp to dynamic graphs (wip)
ggerganov Nov 2, 2023
4fe646f
sync : update tests + fix max op params to 64
ggerganov Nov 2, 2023
83c96d5
sync : ggml-cuda
ggerganov Nov 2, 2023
8401e3e
llama : fix save/load state context size
ggerganov Nov 2, 2023
815f44e
sync : try to fix build on tvOS
ggerganov Nov 2, 2023
16e819d
sync : pass custom graph sizes in training examples
ggerganov Nov 2, 2023
e2349ec
sync : update graph copies to new ggml API
ggerganov Nov 2, 2023
f3fb45b
Merge branch 'master' into sync
ggerganov Nov 2, 2023
7f8e2a5
sync : update sync-ggml.sh with new files
ggerganov Nov 2, 2023
075ee61
Merge branch 'master' into sync
ggerganov Nov 3, 2023
dc22db7
scripts : fix header in sync script
ggerganov Nov 3, 2023
b1592ea
train : fix context size calculations
ggerganov Nov 3, 2023
e50ab5a
llama : increase inference graph size up to 4096 nodes
ggerganov Nov 3, 2023
081a86d
Merge branch 'master' into sync
ggerganov Nov 6, 2023
aa1f36c
Merge branch 'master' into sync
ggerganov Nov 7, 2023
a4de804
train : allocate grads for backward graphs
ggerganov Nov 7, 2023
548ec46
train : allocate grads for gb_tmp
ggerganov Nov 7, 2023
9efc6b9
Merge branch 'master' into sync
ggerganov Nov 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions common/train.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ struct train_state * init_train_state() {
state->opt = new struct ggml_opt_context;
state->opt->ctx = NULL;
state->opt->params = ggml_opt_default_params(GGML_OPT_ADAM);
state->opt->params.graph_size = LLAMA_TRAIN_MAX_NODES;
state->opt->loss_after = 0.0f;

return state;
Expand Down
13 changes: 7 additions & 6 deletions examples/finetune/finetune.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1615,6 +1615,7 @@ int main(int argc, char ** argv) {
opt->params = ggml_opt_default_params(GGML_OPT_ADAM);
opt->params.print_forward_graph = false;
opt->params.print_backward_graph = false;
opt->params.graph_size = LLAMA_TRAIN_MAX_NODES;
opt->params.n_threads = params.common.n_threads;
opt->params.past = params.common.opt_past;
opt->params.delta = params.common.opt_delta;
Expand Down Expand Up @@ -1768,11 +1769,11 @@ int main(int argc, char ** argv) {
for (unsigned order = 0; order < (unsigned) GGML_CGRAPH_EVAL_ORDER_COUNT; ++order) {
ctx_compute = ggml_init(ctx_compute_params);
alloc = ggml_allocr_new_measure(tensor_alignment);
gf = ggml_new_graph(ctx_compute);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = (enum ggml_cgraph_eval_order) order;
gb = ggml_new_graph(ctx_compute);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph(ctx_compute)
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slaren Does this look OK?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, I don't know if gb here needs grads or not.

Copy link
Collaborator

@xaedes xaedes Nov 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gb needs grads, because gb also contains the gf nodes, which have grads.

Changing the bool grads argument from false to true resolves a triggered assert in ggml.c ggml_graph_cpy.

GGML_ASSERT(dst->grads != NULL);

With this change finetune runs, I will report back if the results are good as well.

Copy link
Collaborator

@slaren slaren Nov 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should ggml_graph_cpy be changed to allow skipping the grads if the src has them but not the dst?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was an additional - unrelated to this PR - regression in finetune and train-text-from-scratch due to new yarn rope implementation.

Changing bool grads argument to true and applying #3974 to fix the backward process of rope, the output of finetune is correct.

Copy link

@CoruNethron CoruNethron Nov 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to note, lines 1805 and 1807 below needs that change as well, I missed them at first attempt to copy this fix.
Also, mentioned regression and #3974 fix seems to be critical, because otherwise finetune produces LORA's without any progress from one checkpoint to another.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think gb_tmp also needs the grads=true argument.

: NULL;
loss = llama_build_lora_finetune_graphs(
&model, &lora, alloc, ctx_compute,
Expand Down Expand Up @@ -1801,11 +1802,11 @@ int main(int argc, char ** argv) {
mem_compute_data.resize(max_compute_size);
ctx_compute = ggml_init(ctx_compute_params);
alloc = ggml_allocr_new(mem_compute_data.data(), mem_compute_data.size(), tensor_alignment);
gf = ggml_new_graph(ctx_compute);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = best_order;
gb = ggml_new_graph(ctx_compute);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph(ctx_compute)
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
loss = llama_build_lora_finetune_graphs(
&model, &lora, alloc, ctx_compute,
Expand Down
13 changes: 7 additions & 6 deletions examples/train-text-from-scratch/train-text-from-scratch.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1006,6 +1006,7 @@ int main(int argc, char ** argv) {
opt->params = ggml_opt_default_params(GGML_OPT_ADAM);
opt->params.print_forward_graph = false;
opt->params.print_backward_graph = false;
opt->params.graph_size = LLAMA_TRAIN_MAX_NODES;
opt->params.n_threads = params.common.n_threads;
opt->params.past = params.common.opt_past;
opt->params.delta = params.common.opt_delta;
Expand Down Expand Up @@ -1135,11 +1136,11 @@ int main(int argc, char ** argv) {
for (unsigned order = 0; order < (unsigned) GGML_CGRAPH_EVAL_ORDER_COUNT; ++order) {
ctx_compute = ggml_init(ctx_compute_params);
alloc = ggml_allocr_new_measure(tensor_alignment);
gf = ggml_new_graph(ctx_compute);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = (enum ggml_cgraph_eval_order) order;
gb = ggml_new_graph(ctx_compute);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph(ctx_compute)
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
loss = llama_build_train_graphs(
&model, alloc, ctx_compute,
Expand Down Expand Up @@ -1168,11 +1169,11 @@ int main(int argc, char ** argv) {
mem_compute_data.resize(max_compute_size);
ctx_compute = ggml_init(ctx_compute_params);
alloc = ggml_allocr_new(mem_compute_data.data(), mem_compute_data.size(), tensor_alignment);
gf = ggml_new_graph(ctx_compute);
gf = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, true);
gf->order = best_order;
gb = ggml_new_graph(ctx_compute);
gb = ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false);
gb_tmp = params.common.use_checkpointing
? ggml_new_graph(ctx_compute)
? ggml_new_graph_custom(ctx_compute, LLAMA_TRAIN_MAX_NODES, false)
: NULL;
loss = llama_build_train_graphs(
&model, alloc, ctx_compute,
Expand Down
Loading