Skip to content

Commit

Permalink
quora: reduce needless converters (#189)
Browse files Browse the repository at this point in the history
Because date_time and float are not included in tsv.

Before this change:

```bash
$ time ruby -I lib test/run-test.rb -t "/QuoraDuplicateQuestionPairTest/" -v
Loaded suite test
Started
QuoraDuplicateQuestionPairTest:
  test: #each:                                                                                                                  .: (24.666772)

Finished in 24.667106 seconds.
-------------------------------------------------------------------------------------------------------------------------------------------------
1 tests, 1 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-------------------------------------------------------------------------------------------------------------------------------------------------
0.04 tests/s, 0.04 assertions/s

real    0m25.270s
user    0m24.633s
sys     0m0.451s
```

After this change:

```bash
$ time ruby -I lib test/run-test.rb -t "/QuoraDuplicateQuestionPairTest/" -v
Loaded suite test
Started
QuoraDuplicateQuestionPairTest:
  test: #each:                                                                                                                  .: (17.476536)

Finished in 17.476912 seconds.
-------------------------------------------------------------------------------------------------------------------------------------------------
1 tests, 1 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-------------------------------------------------------------------------------------------------------------------------------------------------
0.06 tests/s, 0.06 assertions/s

real    0m18.117s
user    0m17.554s
sys     0m0.383s
```

Refs #188.
  • Loading branch information
tikkss authored Jan 16, 2024
1 parent 70fcebd commit 3f33534
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion lib/datasets/quora-duplicate-question-pair.rb
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ def open_data
data_path = cache_dir_path + "quora_duplicate_questions.tsv"
data_url = "https://qim.fs.quoracdn.net/quora_duplicate_questions.tsv"
download(data_path, data_url)
CSV.open(data_path, col_sep: "\t", headers: true, converters: :all) do |csv|
CSV.open(data_path, col_sep: "\t", headers: true, converters: :integer) do |csv|
yield(csv)
end
end
Expand Down

0 comments on commit 3f33534

Please sign in to comment.