[path] Avoid unnecessary temp dir renamings in tests

Previously, we kept track of a single global `_test_counter` integer
that was incremented on every call to `mkstemp()` or `mkdtemp()`, and
then the new value was used in the mock temporary file name to ensure
uniqueness. The problem with this is that it led to unnecessary
expectation file changes when adding new `mkdtemp()` or `mkstemp()`
calls to recipes, because doing so would change the names of all
temporary files/directories that were created later in the recipe.

To fix this, keep track of the number of temporary files we've created
for each `prefix` argument (which is the bare minimum necessary to
ensure uniqueness) instead of across all prefixes. This will make it far
less likely that a new `mkdtemp()` or `mkstemp()` call will change the
names of later temporary files, as long as every usage of those
functions in the code uses a different prefix (which is often the case
anyway).

This will use a bit of extra memory: O(number of prefixes used) instead
of O(1). However, this shouldn't be too much of a concern because it
only applies to unit test runs, not to production recipe runs.

Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: chrome_release
Recipe-Nontrivial-Roll: chromiumos
Recipe-Nontrivial-Roll: depot_tools
Recipe-Nontrivial-Roll: fuchsia
Recipe-Nontrivial-Roll: infra
Change-Id: I6e4c8e69ca4dd64a45a53e93ccd965000fd7b7db
Reviewed-on: https://chromium-review.googlesource.com/c/infra/luci/recipes-py/+/2514557
Reviewed-by: Yiwei Zhang <yiwzhang@google.com>
Commit-Queue: Oliver Newman <olivernewman@google.com>
4 files changed
tree: 19ae975e71fdadee3e98564f9f8dc625dc4fa909
  1. doc/
  2. infra/
  3. misc/
  4. recipe_engine/
  5. recipe_modules/
  6. recipe_proto/
  7. recipes/
  8. unittests/
  9. .gitattributes
  10. .gitignore
  11. .style.yapf
  12. .vpython
  13. AUTHORS
  14. codereview.settings
  15. CONTRIBUTORS
  16. LICENSE
  17. OWNERS
  18. PRESUBMIT.py
  19. README.md
  20. README.recipes.md
  21. recipes.py
README.md

Recipes

Recipes are a domain-specific language (embedded in Python) for specifying sequences of subprocess calls in a cross-platform and testable way.

They allow writing build flows which integrate with the rest of LUCI.

Documentation for the recipe engine (including this file!). Take a look at the user guide for some hints on how to get started. See the implementation details doc for more detailed implementation information about the recipe engine.

Contributing

  • Sign the Google CLA.
  • Make sure your user.email and user.name are configured in git config.

Run the following to setup the code review tool and create your first review:

# Get `depot_tools` in $PATH if you don't have it
git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git $HOME/src/depot_tools
export PATH="$PATH:$HOME/src/depot_tools"

# Check out the recipe engine repo
git clone https://chromium.googlesource.com/infra/luci/recipes-py $HOME/src/recipes-py

# make your change
cd $HOME/src/recipes-py
git new-branch cool_feature
# hack hack
git commit -a -m "This is awesome"

# This will ask for your Google Account credentials.
git cl upload -s -r joe@example.com
# Wait for approval over email.
# Click "Submit to CQ" button or ask reviewer to do it for you.
# Wait for the change to be tested and landed automatically.

Use git cl help and git cl help <cmd> for more details.