commit	af414e6a26a8e94d3f2c9833455c2facb347a960	[log] [tgz]
author	Haiyang Pan <hypan@google.com>	Thu Jan 07 17:16:44 2021
committer	LUCI CQ <infra-scoped@luci-project-accounts.iam.gserviceaccount.com>	Thu Jan 07 17:16:44 2021
tree	a971380ccd5395141e06474e90a9dd1354e8af6c
parent	7a94e752699e4b4d2a6a534a55e6e277c9154fd1 [diff]

Update support_3pp recipe module to support more use cases.

Changes includes:
 * Support loading 3pp.pb from a tree structure. The original code
   assumes flat structure which is not very scalable nor flexible.
 * Use the "CIPD package name" rather than package name to identify
   and manage packages.
 * Refine |load_packages_from_path| so that
   * It can load 3pp.pb from a given glob pattern.
   * Compute the hash for the package spec only when it is loaded for
     the first time. With the new tree-based structure, we can not
     assume the specs are stored directly under the package_prefix.
   * Make duplication check optional to support calling the method
    |load_packages_from_path| multiple times.
 * Support setting the experimental mode explicitly.
 * Support setting the source_cache_prefix.
 * Refine |package_prefix| so that
   * It can be empty.
   * Does not end with '/'.
 * Add the link to CIPD instance under the nest step "do upload".
 * Integrate crrev.com/c/2586497 which sets install_mode to 'copy'
   for 3pp source packages.The default on non-Windows is 'symlink'.
   Packages installed via symlinks can't be patched via 'git apply'.

Bug: 1132368, 1152854

Change-Id: I932ea58d71bc11616038ffbca99f211aa4997903
Reviewed-on: https://chromium-review.googlesource.com/c/infra/infra/+/2528117
Commit-Queue: Haiyang Pan <hypan@google.com>
Reviewed-by: Brian Ryner <bryner@google.com>
Cr-Commit-Position: refs/heads/master@{#37374}

24 files changed

tree: a971380ccd5395141e06474e90a9dd1354e8af6c

README.md

infra.git repository

Testing Welcome to the Chrome Infra repository!

Wondering where to start? Check out General Chrome Infrastructure documentation. In particular, to check out this repo and the rest of the infrastructure code, follow the instructions here. The rest of this page is specific to this repo.

Entry points

run.py: wrapper script to run programs contained in subdirectories without having to deal with sys.path modifications.
test.py: multi-purpose script to run tests.
packages/infra_libs/: generally useful functions and classes
infra/services/: standalone programs intended to be run as daemons.
infra/tools: command-line tools, intended to be run by developers.
appengine/: many Chrome-infra-managed AppEngine applications
infra/experimental: for, well, experimental stuff. Once they are stabilized and reviewed, they should be moved in a more permanent place.

Miscellaneous technical stuff

bootstrap/: utilities to set up a proper Python virtual environment.
infra/path_hacks: submodules of this modules give access to modules in the build/ repository. from infra.path_hacks.common import <stg> is actually getting <stg> from build/scripts/common.
utils/: purpose? utils?
Need to bump infra/deployed to pick up changes?
- git push origin <updated hash>:deployed
- mail chrome-troopers@, include:
  - previously deployed hash (for quick rollback)
  - the hash you just pushed
  - the list of CLs that made this push necessary
  - the output of the git push command

Integrating tests with test.py

If you've added a new module, run your tests with test.py:

Create a .coveragerc file in the root directory of the module you want to test. Take a look at another .coveragerc to see what to include in that.
Create a “test” directory in the root directory of the module you want to test. Move your *_test.py files to this directory.

Double-check that your tests are getting picked up when you want them to be: ./test.py test <path-to-package>.

Tests still not getting picked up by test.py? Double-check to make sure you have __init__.py files in each directory of your module so Python recognizes it as a package.

Style

The preferred style is PEP8 with two-space indent; that is, the Chromium Python style, except functions use lowercase_with_underscores. Use yapf (git cl format) to autoformat new code.