summaryrefslogtreecommitdiffstats
path: root/platform_utils.py
Commit message (Collapse)AuthorAgeFilesLines
* Fix EROFS error when root fs is mounted read-onlyEgor Duda2025-04-021-0/+6
| | | | | | | | | | | | | repo attempts to create /etc/.repo_gitconfig.json file, and fails if root file system is mounted read-only. Removing non-existing file on read-only filesystem results in EROFS instead of ENOENT. Bug: 401018409 Change-Id: I64edc0567fb88649f3fd8cacb65a8780744640d4 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/458821 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Egor Duda <egor.duda@gmail.com> Commit-Queue: Egor Duda <egor.duda@gmail.com>
* Remove platform_utils.realpathKaiyi Li2024-03-271-9/+0
| | | | | | | | | | ... since it's just a simple wrapper of os.path.realpath now. Change-Id: I7433e5fe09c64b130f06e2541151dce1961772c9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416637 Tested-by: Kaiyi Li <kaiyili@google.com> Reviewed-by: Greg Edelston <gredelston@google.com> Commit-Queue: Kaiyi Li <kaiyili@google.com>
* Fix drive mounted directory on WindowsKaiyi Li2024-03-271-21/+1
| | | | | | | | | | | | | | On my Windows machine, I mount drive D: to the directory C:\src. The old implementation returns the incorrect 'C:\\??\\Volume{ad2eb15e-f293-4d48-a448-54757d95a97c}' result, which breaks the repo init command. With the use of os.path.realpath, it can return 'D:\\' correctly. Change-Id: Ia5f53989055125cb282d4123cf55d060718aa1ff Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/416580 Reviewed-by: Greg Edelston <gredelston@google.com> Tested-by: Kaiyi Li <kaiyili@google.com> Commit-Queue: Kaiyi Li <kaiyili@google.com>
* cleanup: Update codebase to expect Python 3.6Jason R. Coombs2023-10-311-2/+2
| | | | | | | | | | | - Bump minimum version to Python 3.6. - Use f-strings in a lot of places. Change-Id: I2aa70197230fcec2eff8e7c8eb754f20c08075bb Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/389034 Tested-by: Jason R. Coombs <jaraco@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Commit-Queue: Jason R. Coombs <jaraco@google.com>
* cleanup: leverage yield from in more placesJason R. Coombs2023-10-201-3/+2
| | | | | | | | Change-Id: I4f9cb27d89241d3738486764817b51981444a903 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/390274 Reviewed-by: Aravind Vasudevan <aravindvasudev@google.com> Commit-Queue: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Format codebase with black and check formatting in CQGavin Mak2023-03-221-185/+203
| | | | | | | | | | | | Apply rules set by https://gerrit-review.googlesource.com/c/git-repo/+/362954/ across the codebase and fix any lingering errors caught by flake8. Also check black formatting in run_tests (and CQ). Bug: b/267675342 Change-Id: I972d77649dac351150dcfeb1cd1ad0ea2efc1956 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/363474 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Gavin Mak <gavinmak@google.com> Commit-Queue: Gavin Mak <gavinmak@google.com>
* make file removal a bit more robustMike Frysinger2021-09-281-16/+15
| | | | | | | | | | | | Some of the file removal calls are subject to race conditions (if something else deletes the file), so extend our remove API to have an option to ignore ENOENT errors. Then update a bunch of random call sites to use this new functionality. Change-Id: I31a9090e135452033135337a202a4fc2dbf8b63c Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/319195 Reviewed-by: Sean McAllister <smcallis@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* platform_utils: os.rename exception when src and des on different file systemwenchiching2021-09-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Symptom: repo sync exception Root Cause: os.rename only works when source and destination are on the same file system Solution: using shutil.move to save disk usage, I create links for projects and project-objects, link to folder on another disk lrwxrwxrwx 1 owenwen owenwen 47 Jun 9 16:40 project-objects -> /disk3/AndroidLocalRepos/.repo/project-objects/ lrwxrwxrwx 1 owenwen owenwen 40 Jun 9 16:40 projects -> /disk3/AndroidLocalRepos/.repo/projects/ below are exception I met: """ Traceback (most recent call last): File "/usr/lib/python3.6/multiprocessing/pool.py", line 119, in worker result = (True, func(*args, **kwds)) File "/usr/lib/python3.6/multiprocessing/pool.py", line 44, in mapstar return list(map(*args)) File "/disk2/Android11/.repo/repo/subcmds/sync.py", line 550, in _CheckoutOne project.Sync_LocalHalf(syncbuf, force_sync=force_sync) File "/disk2/Android11/.repo/repo/project.py", line 1251, in Sync_LocalHalf self._InitWorkTree(force_sync=force_sync, submodules=submodules) File "/disk2/Android11/.repo/repo/project.py", line 2801, in _InitWorkTree self._CheckDirReference(self.gitdir, dotgit, share_refs=True) File "/disk2/Android11/.repo/repo/project.py", line 2674, in _CheckDirReference platform_utils.rename(dst_path, src_path) File "/disk2/Android11/.repo/repo/platform_utils.py", line 127, in rename os.rename(src, dst) OSError: [Errno 18] Invalid cross-device link: '/disk2/Android11/system/libhidl/.git/packed-refs' -> '/disk2/Android11/.repo/projects/system/libhidl.git/packed-refs' """ Change-Id: Ifda2f16530cc5a8f280169f482ee858f9e5241d3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/316002 Tested-by: Mike Frysinger <vapier@google.com> Reviewed-by: Mike Frysinger <vapier@google.com>
* platform_utils: delete unused FileDescriptorStreams APIsMike Frysinger2021-02-241-158/+0
| | | | | | | | | | | | | | Now that we've converted the few users of this over to subprocess APIs, we don't need this anymore. It's been a bit hairy to maintain across different operating systems, so there's no desire to bring it back. Using multiprocessing Pool to batch things has been working better in general anyways. Change-Id: I10769e96f60ecf27a80d8cc2aa0d1b199085252e Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/297682 Reviewed-by: Michael Mortensen <mmortensen@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* drop pyversion & is_python3 checkingMike Frysinger2021-01-061-7/+1
| | | | | | | | | | | We're committed to Python 3 at this point, so purge all the is_python3 related dynamic checks. Bug: https://crbug.com/gerrit/10418 Change-Id: I4c8b405d6de359b8b83223c9f4b9c8ffa18ea1a2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292383 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* strip python2-only coding:utf-8 & print_function settingsMike Frysinger2021-01-061-2/+0
| | | | | | | | | | We're committed to Python 3 at this point, so clean up boilerplate. Bug: https://crbug.com/gerrit/10418 Change-Id: Ib1719ba2eb65c53b94881a1a1bf203ddfcaaafed Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/292382 Reviewed-by: Chris Mcdonald <cjmcdonald@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Reland "Port _FileDescriptorStreamsNonBlocking to use poll()"Theodore Dubois2020-03-131-3/+15
| | | | | | | | | | | Now that repo 2 requires Python 3, we can reland this. This reverts commit 91d9587e45608a5f95cd842426b43452a60abb5e. Change-Id: Id5b178ebb53bdba04bfa79cbb5c698ae5080c957 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/258672 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Theodore Dubois <tbodt@google.com>
* platform_utils: have Windows select stream return "" at EOFMike Frysinger2020-02-181-1/+1
| | | | | | | | | | This matches *NIX behavior where the last read is '', not None. Bug: https://crbug.com/gerrit/12329 Change-Id: I48b026b4d1b8d7c6abbce198757b970931869e1a Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/255352 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* Fix blank line issues reported by flake8David Pursehouse2020-02-121-0/+4
| | | | | | | | | | | | | | | | | | | | - E301 expected 1 blank line - E302 expected 2 blank lines - E303 too many blank lines - E305 expected 2 blank lines after class or function definition - E306 expected 1 blank line before a nested definition Fixed automatically with autopep8: git ls-files | grep py$ | xargs autopep8 --in-place \ --select E301,E302,E303,E305,E306 Manually fix issues in project.py caused by misuse of block comments. Change-Id: Iee840fcaff48aae504ddac9c3e76d2acd484f6a9 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/254599 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: David Pursehouse <dpursehouse@collab.net>
* Revert "Port _FileDescriptorStreamsNonBlocking to use poll()"v1.13.9.1Mike Frysinger2020-02-031-15/+3
| | | | | | | | | | | | | | | | This reverts commit 1e01a7444536b4865feb94c398b68a936a463ddc. Not all platforms support select.poll() currently it seems. At least macOS's Python 2 doesn't (while macOS Python 3 does). Lets back this out for the existing release series and once we start repo-2 which is Python 3-only, we can put this back in. Change-Id: I205206b0fa4fe2d755f4fbc6ec683ad125f27cc2 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/253072 Reviewed-by: Michael Mortensen <mmortensen@google.com> Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Mike Frysinger <vapier@google.com>
* Fix method signature of platform_utils.FileDescriptorStreams._create_stream()Rostislav Krasny2020-01-251-1/+1
| | | | | | | Change-Id: Ib80e4ec5e540d97488e7564703ddbcb74350fdfd Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/251836 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Rostislav Krasny <rostigm@gmail.com>
* Port _FileDescriptorStreamsNonBlocking to use poll()Theodore Dubois2019-12-181-3/+15
| | | | | | | | | | | select() has a limit of FD_SETSIZE file descriptors. If you run repo sync -j500 you'll pretty quickly hit this limit and get "file descriptor out of range for select" errors. poll() has no such limit. Change-Id: I21f350e472bda1db03dcbcc437645c23dbc7a901 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/248852 Reviewed-by: Mike Frysinger <vapier@google.com> Tested-by: Theodore Dubois <tbodt@google.com>
* sync: make .git init more robustMike Frysinger2019-11-121-4/+5
| | | | | | | | | | | | | | | | Hitting Ctrl-C in the middle of this func will leave the .git in a bad state that requires manual recovery. The code tries to catch all exceptions and recover by deleting the incomplete .git dir, but it omits KeyboardInterrupt which Exception misses. We could add that to the recovery path, but we can make this more robust with a different approach: set up everything in .git.tmp/ and only move it to .git/ once we've fully initialized it. Change-Id: I0f5b97f2e19fc39cffc3e5e23993a2da7220f4e3 Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/244733 Reviewed-by: David Pursehouse <dpursehouse@collab.net> Tested-by: Mike Frysinger <vapier@google.com>
* set default file encoding to utf-8Mike Frysinger2019-06-131-0/+1
| | | | | | | | There's no reason to support any other encoding in these files. This only affects the files themselves and not streams they open. Bug: https://crbug.com/gerrit/10418 Change-Id: I053cb40cd3666ce5c8a0689b9dd938f24ca765bf
* platform_utils: Fix exception handling in _walk_windows_implDavid Pursehouse2018-11-061-1/+1
| | | | Change-Id: I6b79cbc4c1bbbe17ffe8361fe1544434beaa9059
* Add support for long pathsRenaud Paquay2018-10-221-16/+110
| | | | | | | | | | | | | | | | | | | | * Add more file i/o wrappers in platform_utils to allow using long paths (length > MAX_PATH) on Windows. * Paths using the long path syntax ("\\?\" prefix) should never escape the platform_utils API surface area, so that this specific syntax is not visible to the rest of the repo code base. * Forward many calls from os.xxx to platform_utils.xxx in various place to ensure long paths support, specifically when repo decides to delete obsolete directories. * There are more places that need to be converted to support long paths, this commit is an initial effort to unblock a few common use cases. * Also, fix remove function to handle directory symlinks Change-Id: If82ccc408e516e96ff7260be25f8fd2fe3f9571a
* fix some sync error while using python3Dylan Deng2018-07-241-1/+6
| | | | Change-Id: I70925e48756c356d48359679d8ad1b9e33a68595
* Replace all os.remove callsRenaud Paquay2017-08-311-0/+17
| | | | | | | | os.remove raises an exception when deleting read-only files on Windows. Replace all calls with calls to platform_utils.remove, which deals with deals with that issue. Change-Id: I4dc9e0c9a36b4238880520c69f5075eca40f3e66
* Implement islink, readlink and realpath using Win32 apiRenaud Paquay2017-08-311-0/+54
| | | | Change-Id: I18452cbb32d24db73601ad10485dbe6bb278731c
* Port os.rename calls to work on WindowsRenaud Paquay2017-05-291-0/+17
| | | | | | | | os.rename fails on Windows if the destination exists, so replace os.rename to platform_utils.rename which handles the platform differences. Change-Id: I15a86f10f65eedee5b003b80f88a0c28a3e1aa48
* Workaround shutil.rmtree limitation on WindowsRenaud Paquay2017-05-291-0/+15
| | | | | | | | | | By default, shutil.rmtree raises an exception when deleting readonly files on Windows. Replace all shutil.rmtree with platform_utils.rmtree, which adds an error handler to make files read-write when they can't be deleted. Change-Id: I9cfea9a7b3703fb16a82cf69331540c2c179ed53
* Add support for creating symbolic links on WindowsRenaud Paquay2017-05-291-0/+43
| | | | | | | | | | | | Replace all calls to os.symlink with platform_utils.symlink. The Windows implementation calls into the CreateSymbolicLinkW Win32 API, as os.symlink is not supported. Separate the Win32 API definitions into a separate module platform_utils_win32 for clarity. Change-Id: I0714c598664c2df93383734e609d948692c17ec5
* Make "git command" and "forall" work on WindowsRenaud Paquay2017-05-291-0/+169
Python on Windows does not support non blocking file operations. To workaround this issue, we instead use Threads and a Queue to simulate non-blocking calls. This is happens only when running with the native Windows version of Python, meaning Linux and Cygwin are not affected by this change. Change-Id: I4ce23827b096c5138f67a85c721f58a12279bb6f
generated by cgit v1.2.3-54-g00ecf (git 2.39.0) at 2025-05-14 09:58:56 +0000