Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
watch: add file delete/rename handling #10386
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
milas
requested review from
StefanScherer,
glours,
laurazard,
ndeloof,
nicksieger and
ulyssessouza
and removed request for
a team
ndeloof
approved these changes
Mar 21, 2023
|
@milas you forgot to sign you commit! I enabled the auto-merge, this PR is just waiting for your commit's update 😉 |
This approach mimics Tilt's behavior[^1]: 1. At sync time, `stat` the path on host 2. If the path does not exist -> `rm` from container 3. If the path exists -> sync to container By handling things this way, we're always syncing based on the true state, regardless of what's happened in the interim. For example, a common pattern in POSIX tools is to create a file and then rename it over an existing file. Based on timing, this could be a sync, delete, sync (every file gets seen & processed) OR a delete, sync (by the the time we process the event, the "temp" file is already gone, so we just delete it from the container, where it never existed, but that's fine since we deletes are idempotent thanks to the `-f` flag on `rm`). Additionally, when syncing, if the `stat` call shows it's for a directory, we ignore it. Otherwise, duplicate, nested copies of the entire path could get synced in. (On some OSes, an event for the directory gets dispatched when a file inside of it is modified. In practice, I think we might want this pushed further down in the watching code, but since we're already `stat`ing the paths here now, it's a good place to handle it.) Lastly, there's some very light changes to the text when it does a full rebuild that will list out the (merged) set of paths that triggered it. We can continue to improve the output, but this is really helpful for understanding why it's rebuilding. [^1]: https://github.com/tilt-dev/tilt/blob/db7f887b0658ed042069dc0ff4cb266fe0596c23/internal/controllers/core/liveupdate/reconciler.go#L911 Signed-off-by: Milas Bowman <milas.bowman@docker.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What I did
Add support for file deletes & renames while running
alpha watch.This approach mimics Tilt's behavior1:
statthe path on hostrmfrom containerBy handling things this way, we're always syncing based on the true state, regardless of what's happened in the interim. For example, a common pattern in POSIX tools is to create a file and then rename it over an existing file. Based on timing, this could be a sync, delete, sync (every file gets seen & processed) OR a delete, sync (by the the time we process the event, the "temp" file is already gone, so we just delete it from the container, where it never existed, but that's fine since we deletes are idempotent thanks to the
-fflag onrm).Additionally, when syncing, if the
statcall shows it's for a directory, we ignore it. Otherwise, duplicate, nested copies of the entire path could get synced in. (On some OSes, an event for the directory gets dispatched when a file inside of it is modified. In practice, I think we might want this pushed further down in the watching code, but since we're alreadystating the paths here now, it's a good place to handle it.)Lastly, there's some very light changes to the text when it does a full rebuild that will list out the (merged) set of paths that triggered it. We can continue to improve the output, but this is really helpful for understanding why it's rebuilding.
Related issue
JIRA: ENV-139
(not mandatory) A picture of a cute animal, if possible in relation to what you did
Footnotes
https://github.com/tilt-dev/tilt/blob/db7f887b0658ed042069dc0ff4cb266fe0596c23/internal/controllers/core/liveupdate/reconciler.go#L911 ↩