Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refine some file I/O functions' doxygen comments #549

Merged
merged 1 commit into from
Jan 9, 2024
Merged

Refine some file I/O functions' doxygen comments #549

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 34 additions & 13 deletions include/matx/core/file_io.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,27 @@ using namespace pybind11::literals;


/**
* Read a CSV file into a tensor view
* @brief Read a CSV file into a tensor view
*
* CSVs are currently read in using the Python interpreter through pybind11.
*This has a startup performance hit, but CSV reading is intended to be a
*slow-path function, so this is not a critical component to speed up. Currently
*1D and 2D tensors are supported only.
* This has a startup performance hit, but CSV reading is intended to be a
* slow-path function, so this is not a critical component to speed up. Currently
* 1D and 2D tensors are supported only.
*
* @tparam TensorType
* Data type of tensor
* @param t
* Tensor to read data into
* @param fname
* File path of .csv file
* @param delimiter
* Delimiter to use for CSV file
* @param skip_header
* Skip the header row of the CSV file, default as `true`.
**/
template <typename TensorType>
void read_csv(TensorType &t, const std::string fname,
const std::string delimiter, bool header = true)
const std::string delimiter, bool skip_header = true)
{
MATX_NVTX_START("", matx::MATX_NVTX_LOG_API)

Expand All @@ -146,18 +157,27 @@ void read_csv(TensorType &t, const std::string fname,

auto np = pybind11::module_::import("numpy");
auto obj = np.attr("genfromtxt")("fname"_a = fname.c_str(), "delimiter"_a = delimiter,
"skip_header"_a = header ? 1 : 0,
"skip_header"_a = skip_header,
"dtype"_a = detail::MatXPybind::GetNumpyDtype<typename TensorType::scalar_type>());
pb->NumpyToTensorView(t, obj);
}

/**
* Read a CSV file into a tensor view
* Write a CSV file from a tensor view
*
* CSVs are currently read in using the Python interpreter through pybind11.
*This has a startup performance hit, but CSV reading is intended to be a
*slow-path function, so this is not a critical component to speed up. Currently
*1D and 2D tensors are supported only.
* CSVs are currently written using the Python interpreter through pybind11.
* This has a startup performance hit, but CSV writing is intended to be a
* slow-path function, so this is not a critical component to speed up. Currently
* 1D and 2D tensors are supported only.
*
* @tparam TensorType
* Data type of tensor
* @param t
* Tensor to write data from
* @param fname
* File path of .csv file
* @param delimiter
* Delimiter to use for CSV file
**/
template <typename TensorType>
void write_csv(const TensorType &t, const std::string fname,
Expand Down Expand Up @@ -220,7 +240,7 @@ void read_mat(TensorType &t, const std::string fname,
}

/**
* @brief Read a MAT file into a tensor view
* @brief Read a MAT file and return a tensor view
*
* MAT files use SciPy's loadmat() function to read various MATLAB file
* types in. MAT files are supersets of HDF5 files, and are allowed to
Expand All @@ -232,7 +252,8 @@ void read_mat(TensorType &t, const std::string fname,
* File name of .mat file
* @param var
* Variable name inside of .mat to read
*
* @return
* Tensor view of data read from file
**/
template <typename TensorType>
auto read_mat(const std::string fname,
Expand Down