Skip to content

Commit

Permalink
Fix typo in name of hybrid engine function (#3704)
Browse files Browse the repository at this point in the history
* Fix typo in name of hybrid engine function

* Fix
  • Loading branch information
loadams authored Jun 8, 2023
1 parent 0977106 commit fc8e5c8
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions deepspeed/module_inject/containers/features/split_qkv.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,14 +123,14 @@ def reset_qkv_experimental(self):
for data in qkv_data:
del data

def set_attn_parameters_wo_copy(self, Z3_enabled=False):
def set_attn_params_wo_copy(self, Z3_enabled=False):
self.module.attention.attn_ow = self.dense_w
self.module.attention.attn_ob = self.dense_b
if not Z3_enabled:
# In initialize_tensors, we create a fused qkvw with the appropriate shape
# and copy the qw, qb, kw, kb, vw, vb into it
self.module.attn_qkvw = self.qkvw
self.module.attn_qkvb = self.qkvb
self.module.attention.attn_qkvw = self.qkvw
self.module.attention.attn_qkvb = self.qkvb

# We reset the data for qw (which is the original model parameter) to point
# to the fused weight matrix we have created here
Expand Down

0 comments on commit fc8e5c8

Please sign in to comment.