Skip to content

Commit

Permalink
Add shapelib and Torch ODS gen tests
Browse files Browse the repository at this point in the history
Also update the scripts to return 1 when there are changes. The
tests can check and report if there are any changes.

TEST:
The passing tests look like:
:::: Run shapelib update tests
Shape Lib is up to date
:::: Run torch_ods update tests
Torch ODS is up to date

Failing would like like:

anush@sdp ~/github/torch-mlir % time ./build_tools/update_shape_lib.sh
PYTHONPATH=/home/anush/github/torch-mlir/build/tools/torch-mlir/python_packages/torch_mlir python -m torch_mlir.dialects.torch.importer.jit_ir.build_tools.shape_lib_gen --pytorch_op_extensions= --torch_transforms_cpp_dir=/home/anush/github/torch-mlir/lib/Dialect/Torch
/Transforms
Generated files are not up to date (see diff below)
>>> Please run ./build_tools/update_torch_ods.sh and ./build_tools/update_shape_lib.sh <<<
diff --git a/lib/Dialect/Torch/Transforms/ShapeLibrary.cpp b/lib/Dialect/Torch/Transforms/ShapeLibrary.cpp
index f7a8f69c..a780ee33 100644
--- a/lib/Dialect/Torch/Transforms/ShapeLibrary.cpp
+++ b/lib/Dialect/Torch/Transforms/ShapeLibrary.cpp
@@ -14,6 +14,8 @

 #include "torch-mlir/Dialect/Torch/Transforms/Passes.h"

+foo
+
 using namespace mlir;

 StringRef mlir::torch::Torch::getShapeLibrary() {
./build_tools/update_shape_lib.sh  0.02s user 0.01s system 104% cpu 0.035 total
  • Loading branch information
powderluv committed Aug 31, 2022
1 parent e52e886 commit 2bffba6
Show file tree
Hide file tree
Showing 3 changed files with 34 additions and 0 deletions.
16 changes: 16 additions & 0 deletions build_tools/python_deploy/build_linux_packages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,22 @@ function test_in_tree() {
cd /main_checkout/torch-mlir/
export PYTHONPATH="/main_checkout/torch-mlir/build/tools/torch-mlir/python_packages/torch_mlir"

echo ":::: Run shapelib update tests"
if ! ./build_tools/update_shape_lib.sh; then
echo Shape Lib is out of date with the installed PyTorch version
exit 1
else
echo Shape Lib is up to date
fi

echo ":::: Run torch_ods update tests"
if ! ./build_tools/update_torch_ods.sh; then
echo Torch ODS is out of date with the installed PyTorch version
exit 1
else
echo Torch ODS is up to date
fi

echo ":::: Run refbackend e2e integration tests"
python -m e2e_testing.main --config=refbackend -v

Expand Down
9 changes: 9 additions & 0 deletions build_tools/update_shape_lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,12 @@ PYTHONPATH="${pypath}" python \
-m torch_mlir.dialects.torch.importer.jit_ir.build_tools.shape_lib_gen \
--pytorch_op_extensions=${ext_module:-""} \
--torch_transforms_cpp_dir="${torch_transforms_cpp_dir}"

if ! $(git diff --quiet ./lib/Dialect/Torch/Transforms/ShapeLibrary.cpp); then
echo "#######################################################"
echo "Generated files are not up to date (see diff below)"
echo ">>> Please verify you are running the latest PyTorch nightly or source build and if required push these changes up for review <<<"
echo "#######################################################"
git diff --color=always ./lib/Dialect/Torch/Transforms/ShapeLibrary.cpp
exit 1
fi
9 changes: 9 additions & 0 deletions build_tools/update_torch_ods.sh
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,12 @@ PYTHONPATH="${pypath}" python \
--torch_ir_include_dir="${torch_ir_include_dir}" \
--pytorch_op_extensions="${ext_module}" \
--debug_registry_dump="${torch_ir_include_dir}/JITOperatorRegistryDump.txt"

if ! $(git diff --quiet ./include/torch-mlir/Dialect/Torch/IR/GeneratedTorchOps.td); then
echo "#######################################################"
echo "Generated files are not up to date (see diff below)"
echo ">>> Please verify you are running the latest PyTorch nightly or source build and if required push these changes up for review <<<"
echo "#######################################################"
git diff --color=always ./include/torch-mlir/Dialect/Torch/IR/GeneratedTorchOps.td
exit 1
fi

0 comments on commit 2bffba6

Please sign in to comment.