Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 0.6.0 - PyFileSystem2 support #327

Merged
merged 168 commits into from
Sep 10, 2019
Merged
Changes from 1 commit
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
b640cbd
:sparkles: initial support of pythonfilesystem2 in data loader
chfw Jul 21, 2019
1889142
:sparkles: use pythonfilesystem2 to write bytes out
chfw Jul 21, 2019
3ec168a
:bug: convert ascii to unicode in python 2
chfw Jul 22, 2019
d800ef8
:green_heart: fix linting issue
chfw Jul 22, 2019
9249ed3
:green_heart: fix file writing
chfw Jul 22, 2019
03c3d4b
:fire: ignore unicode warning
chfw Jul 22, 2019
9755315
:microscope: update context with fs.path.join
chfw Jul 23, 2019
daf6956
:green_heart: updated all os.path with fs.path
chfw Jul 23, 2019
e1322e4
:lipstick: update coding style
chfw Jul 23, 2019
db5f90a
:hammer: more replacement in test targets
chfw Jul 23, 2019
d723fe1
:tractor: further curb of os.path
chfw Jul 23, 2019
b921ffe
:green_heart: upgrade engine tests
chfw Jul 23, 2019
e4f1776
:hammer: more replacement of os.path -> fs.path
chfw Jul 23, 2019
52b1773
:hammer: update fs layer
chfw Jul 23, 2019
4822b83
:hammer: bank the template refactoring change
chfw Jul 23, 2019
b78e655
:green_heart: all existing tests now pass
chfw Jul 23, 2019
45bf54a
:sparkles: update constants.py
chfw Jul 23, 2019
46a018d
:green_heart: fix linux unit test failures
chfw Jul 23, 2019
dc86afb
:green_heart: further fight with unit tests
chfw Jul 23, 2019
cdabe6e
:fire: remove os.path.join
chfw Jul 23, 2019
b86fc71
:green_heart: remore os.path.join in repo.py and :hammer: code refact…
chfw Jul 24, 2019
ba734b5
:hammer: pin down lowerest version of fs
chfw Jul 24, 2019
a7f6349
:hammer: more replacement in tests
chfw Jul 24, 2019
b8033ad
:sparkles: use migrated moban-handlebars for testing
chfw Jul 24, 2019
714ed5b
:bug: use normcase() to get backward slashes
chfw Jul 24, 2019
2385a02
:green_heart: fix final bug in windows os path
chfw Jul 24, 2019
22cd1a8
:green_heart: fix the misunderstanding of fs.path.join
chfw Jul 24, 2019
48d0754
:green_heart: one more attempt to fix windows behaviour
chfw Jul 24, 2019
98b6f8d
:green_heart: fix the wild behavior
chfw Jul 24, 2019
2481464
:green_heart: use norm path over norm case
chfw Jul 24, 2019
1f94f00
:sparkles: nailed the final bug
chfw Jul 24, 2019
9598581
:sparkles: python file system 2 is driving downstream code mad
chfw Jul 24, 2019
32e097a
:sparkles: no longer I know which file path is correct
chfw Jul 24, 2019
283fe2b
:art: yes, last one should work
chfw Jul 24, 2019
bb9e261
:shirt: fix lint errors
chfw Jul 24, 2019
fef52fc
:short: :hammer: code refactoring
chfw Jul 24, 2019
0d14f32
:shirt: fix last lint error
chfw Jul 24, 2019
989a2c1
:hammer: code refactoring
chfw Jul 25, 2019
8ab0258
:books: update change log
chfw Jul 25, 2019
2281a4f
:hammer: code refactoring
chfw Jul 25, 2019
438d6de
:shirt: pin up python file system to 2.2.0
chfw Jul 25, 2019
400035a
:fire: minor tuning
chfw Jul 25, 2019
dfd660e
:lipstick: pump up version number
chfw Jul 26, 2019
b3d427c
:sparkles: python file system compliant file openers
chfw Jul 26, 2019
bc41a95
:tractor: relocate level 9 docs to deprecated but kept for unit tests
chfw Jul 26, 2019
9a0bc5f
:microscope: regression test new pypi syntax
chfw Jul 26, 2019
41e1772
:lipstick: update coding style
chfw Jul 26, 2019
45c3b9c
:tractor: deprecate old level 10 git repo syntax
chfw Jul 26, 2019
3bb0569
:microscope: regression test repo://repo_name/path
chfw Jul 26, 2019
5362983
:green_heart: install the python file system entry points
chfw Jul 26, 2019
6a16f58
:newspaper: add missing files
chfw Jul 26, 2019
9511469
:bug: fix windows bug c:/ , where c is found to be a pypi repo
chfw Jul 26, 2019
0632da3
:green_heart: make test
chfw Jul 26, 2019
f5ec738
:bug: min requirement for appdir has to be 1.4.3, fs 2.2.0 requires 1…
chfw Jul 26, 2019
45ceaf5
:fire: deprecated old syntax. and add read_unicode for moban-handlebars
chfw Jul 27, 2019
904324a
:books: update change log
chfw Jul 27, 2019
d8ea619
:newspaper: address unicode related review comments
chfw Jul 27, 2019
08ec831
:hammer: use u for unicode convertion and :microscope: log errors fro…
chfw Jul 27, 2019
3b1e781
:bug: return directory as it is if it is FSURL. need to assume all ur…
chfw Jul 27, 2019
8b24e83
:bug: avoid circular import
chfw Jul 27, 2019
6c1d512
:green_heart: make flake8 happy
chfw Jul 27, 2019
61b4835
:sparkles: add templates in a zip demo. resolves #305
chfw Jul 27, 2019
6a7b506
:shirt: update coding style. will do code refactoring later
chfw Jul 27, 2019
04dd0dc
:fire: use registry.install instead of setup.py entry points
chfw Jul 27, 2019
6255fe2
:bug: install master branch of custom jinja2-fsloader
chfw Jul 27, 2019
818ea5b
:hammer: code refactoring
chfw Jul 28, 2019
7f20b1b
:bug: tuning the sequence of rnd installation
chfw Jul 28, 2019
caf3d98
:shirt: code reformatting and :bug: fix python 2 unicode problem
chfw Jul 28, 2019
a0bd87f
:hammer: convert all file names at entry to unicode
chfw Jul 28, 2019
71c8fec
:bug: fix get_template non unicode problem
chfw Jul 28, 2019
45a6bd4
:shirt: update coding style
chfw Jul 28, 2019
150e6aa
:hammer: backward compactibility
chfw Jul 28, 2019
c6cf5c7
:shirt: coding style update
chfw Jul 28, 2019
4462394
:sparkles: use EnhancedOSFS for custom pypi and repo
chfw Jul 28, 2019
7623738
:shirt: fix linting errors
chfw Jul 28, 2019
a1863a1
:sparkles: enhance tar fs to have geturl function
chfw Jul 28, 2019
9f0c3ca
:shirt: update coding style
chfw Jul 28, 2019
36cf3fd
:books: level 20 will provide the data in an alien file system. resol…
chfw Jul 28, 2019
61ee49e
:books: write files to an alien file system. resolves #306
chfw Jul 28, 2019
efa9eb1
:shirt: update coding style
chfw Jul 28, 2019
44a1d75
:bug: avoid create a zip folder
chfw Jul 28, 2019
374c2e5
:bug: unicode bug again in python 2
chfw Jul 28, 2019
a38e141
:bug: inverse the logic for making windows dirs
chfw Jul 28, 2019
d56d2e6
:books: copy from zip to zip file. resolves #307. :sparkles: write mu…
chfw Jul 29, 2019
9ba3c84
:newspaper: add missing source file
chfw Jul 29, 2019
1c1d5bf
:ambulance: python 2 non-unicode string caught again by pyfilesystem2…
chfw Jul 29, 2019
a215576
:bug: unicode appears in python2 and but we write bytes in python2
chfw Jul 30, 2019
8d816a6
:microscope: initial unit test for moban file system layer
chfw Jul 31, 2019
6862d0e
:microscope: add file existence test
chfw Jul 31, 2019
579da67
:shirt: coding style update
chfw Jul 31, 2019
1d928ba
:bug: fs 2.2.0 is too low version to get current moban working
chfw Jul 31, 2019
fe42c43
:shirt: pyfs2 on python 2 caught non-unicode again. #303
chfw Jul 31, 2019
754e1a2
:microscope: full test suite for file system py
chfw Jul 31, 2019
7bc6165
:green_heart: make unit test pass
chfw Jul 31, 2019
b5db605
:green_heart: make windows build pass
chfw Aug 1, 2019
654692b
:hammer: code refactor file system py
chfw Aug 1, 2019
b26ceef
:hammer: further refactoring to reduce the length of file system
chfw Aug 1, 2019
6ff7499
:bug: fix python 2 compactibility. urlparse #303
chfw Aug 1, 2019
e1f4efa
:tractor: further shrink the file system
chfw Aug 1, 2019
adef813
:sparkles: pyfs2 review feedback for better zip/tar fs url. https://g…
chfw Aug 4, 2019
40ee461
:shirt: update coding style
chfw Aug 4, 2019
b7948aa
:fire: use jinja2-fsloader as it is. https://github.com/althonos/jinj…
chfw Aug 4, 2019
1413e7d
:fire: remove internal pypifs opener with external and universal pypi…
chfw Aug 5, 2019
46f5842
book: update documenation
chfw Aug 6, 2019
ec7f375
:art: install rnd version of fs2 for debugging windows build failure
chfw Aug 6, 2019
9af4004
:art: try use local adapted version of osfs
chfw Aug 6, 2019
65fac55
:ambulance: it looks like pypifs have to pin on the next version of p…
chfw Aug 6, 2019
32f5eba
:fire: remove adapation code and use next version of pyfilesystem2. h…
chfw Aug 7, 2019
85ea8ac
:ambulance: use next version of jinja2-fsloader. https://github.com/a…
chfw Aug 7, 2019
91198d1
:fire: temporarily disable min requirement test beacause min requirem…
chfw Aug 7, 2019
485e4f2
:shirt: code formatting
chfw Aug 7, 2019
525c68b
:shirt: use released version jinja2-fsloader. https://github.com/alth…
chfw Aug 7, 2019
eaf5d7b
:hammer: code refactoring
chfw Aug 8, 2019
641b5a0
:tractor: switch the lane from file:// to osfs://. https://github.com…
chfw Aug 8, 2019
4aec262
:shirt: update coding style
chfw Aug 8, 2019
73155cc
:green_heart: revert unwanted change and make windows unit tests pass
chfw Aug 8, 2019
22ec2ba
:fire: remove duplicated codes
chfw Aug 8, 2019
609c435
:hammer: replace custom join with url_join
chfw Aug 8, 2019
d38a5f4
:hammer: optimize the code
chfw Aug 8, 2019
7061996
:hammer: minor update on checking filename
chfw Aug 9, 2019
ad459b6
:books: update yaml indentation
chfw Aug 9, 2019
28a6b80
:green_heart: :hammer: code refactor utils.py
chfw Aug 9, 2019
03acbc5
:fire: remove unwanted, useless changes occured in development stage.…
chfw Aug 9, 2019
e26c050
:fire: remove unwanted changes.
chfw Aug 9, 2019
bc8ec03
:bug: fix the regression error where hash store is not disabled becau…
chfw Aug 9, 2019
8a57815
:green_heart: update hash store unit test to be windows friendly
chfw Aug 9, 2019
4a55353
:green_heart: fix the path again
chfw Aug 9, 2019
3f57a32
:green_heart: fix the typo
chfw Aug 9, 2019
5a8281a
:microscope: more tests for buffered writer
chfw Aug 9, 2019
9187852
:hammer: code refactoring
chfw Aug 10, 2019
483d11f
:shirt: update coding style
chfw Aug 10, 2019
6f79615
:hammer: further integration with pyfs2
chfw Aug 10, 2019
dd51c3c
:shirt: update coding style
chfw Aug 10, 2019
a8c647b
:hammer: minor code tuning
chfw Aug 10, 2019
48578ac
:hammer: code refactoring. one place to import fs
chfw Aug 10, 2019
4958b62
:green_heart: make the tests pass
chfw Aug 10, 2019
7ab2d29
:hammer: further collapse the utils.py code get_template_path
chfw Aug 10, 2019
02823a5
:green_heart: make python 2 pass as multi_fs hates str
chfw Aug 10, 2019
ff45d40
:fire: replace file searching among multiple dirs using multi-fs
chfw Aug 10, 2019
860dedf
:fire: remove unused imports. flake8 warns
chfw Aug 10, 2019
c867b73
:tractor: relocate search_file function
chfw Aug 10, 2019
b31c322
:tractor: code coherence, moving code to where it has stronger connec…
chfw Aug 10, 2019
c4e9cde
:shirt: coding style update
chfw Aug 10, 2019
123b0ab
:fire: replace mult directory search with multi_fs whic exists already
chfw Aug 10, 2019
fcb6d6e
:shirt: coding style update
chfw Aug 10, 2019
9073863
:tractor: function coherence, file permissions belong to file system.…
chfw Aug 10, 2019
e3d7d74
:shirt: coding style
chfw Aug 10, 2019
1316c3b
:green_heart: fix python2 again. non-unicode string
chfw Aug 10, 2019
c556482
:fire: remove unused import
chfw Aug 10, 2019
411283f
:shirt: coding style update using black
chfw Aug 10, 2019
f5f6206
:sparkles: update api interface using fs.multifs.MultiFS instance
chfw Aug 11, 2019
20282f8
:fire: remove unused import
chfw Aug 11, 2019
a0db5d2
:books: update docstring and extension guide
chfw Aug 11, 2019
6ea5cb8
:shirt: coding style update
chfw Aug 11, 2019
38a3826
Merge pull request #302 from moremoban/pyfs2
chfw Aug 11, 2019
8815f7a
gitfs2 - the final syntax for repo (#312)
chfw Aug 16, 2019
96aa2f9
Fix any other regressions (#313)
chfw Aug 18, 2019
426a10d
Create FUNDING.yml
chfw Aug 18, 2019
adfafe9
Tidy FUNDING.yml (#315)
jayvdb Aug 19, 2019
8fb867a
Code refactoring (#316)
chfw Aug 20, 2019
e5c26b0
:hammer: :tractor: refactored tests (#317)
chfw Aug 21, 2019
5d3e2ae
Code cleanning (#318)
chfw Aug 21, 2019
df830b6
:bug: update templated count (#320)
chfw Aug 22, 2019
930a34f
Extended override (#321)
chfw Aug 23, 2019
9d28ccd
:books: mentioned about git commit message emojis. fix #279
chfw Aug 23, 2019
ff18b41
-D hello=world is added. (#324)
chfw Aug 29, 2019
bf5ab52
Debug option -v and -vv (#326)
chfw Sep 9, 2019
c62ea15
:egg: :ferris_wheel: release 0.6.0 - PyFileSystem2 Support
chfw Sep 9, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
🔥 minor tuning
  • Loading branch information
chfw committed Jul 25, 2019

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 400035a5036cd5764c9bc7b2217e31efae7f1fc8
8 changes: 2 additions & 6 deletions moban/mobanfile/templates.py
Original file line number Diff line number Diff line change
@@ -60,14 +60,10 @@ def _listing_directory_files_recusively(source, actual_source_path, dest):
src_file_under_dir = file_system.path_join(source, file_name)
dest_file_under_dir = file_system.path_join(dest, file_name)
real_src_file = file_system.path_join(actual_source_path, file_name)
if file_system.is_file(
file_system.path_join(actual_source_path, file_name)
):
if file_system.is_file(real_src_file):
template_type = _get_template_type(src_file_under_dir)
yield (src_file_under_dir, dest_file_under_dir, template_type)
elif file_system.is_dir(
file_system.path_join(actual_source_path, file_name)
):
elif file_system.is_dir(real_src_file):
for a_triple in _listing_directory_files_recusively(
src_file_under_dir, real_src_file, dest_file_under_dir
):