commit 0006f4737647ad0790431737586fd700085f8b0e
author David 'Digit' Turner <digit@google.com> Tue Aug 07 07:12:36 2018
committer Commit Bot <commit-bot@chromium.org> Tue Aug 07 07:12:36 2018
tree 4397a127805c0c2fa703b820fcdc56bcd23982cd
parent 66101cc52c18b04c85e87239a1fc0ae5c4297e6e

android: build: Use .pydeps file to store Python dependencies.

NOTE: This is a reland of the following CL which was reverted:

  https://chromium-review.googlesource.com/c/chromium/src/+/1131190

The revert was due to a problem with an internal clank/DEPS runhooks
step that calls jinja_template.py (see BUG entry below for details).
The new CL was checking inputs by defaults, which would raise an
error during gclient sync.

To ease review, the first CL on gerrit is the original one,
unmodified, and the second fixes the issue.

---- end of NOTE ------

ProTip: start by looking at the action_with_pydefps() definition
in internal_rules.gni to review changes in this CL.

This CL tries to make "gn analyze" smarter by ensuring that
it knows about Python module dependencies, for any python
script under //build.

At the moment, these dependencies are computed by
build_utils.WriteDepFile() automatically (unless the
add_pydeps=False argument is passed). This adds all imported
module paths to the generated .d file written by the script.

Unfortunately, because this .d file is generated by the
actions, the corresponding Python sources are not part of
the target's inputs, known by GN at parse time, and thus do
not appear in "gn analyze" properly.

This CL tries to solve the problem by adding a new template
named "action_with_pydeps", which acts as "action", but also
expects a foo.pydeps file for every foo.py invoked through
the 'script' scope variable.

This '.pydeps' file is read directly at GN parse time and
added to the action's target inputs.

These .pydeps contain the path of each imported module, one
per line, and are generated with build/print_python_deps.py

Benchmarking shows no significant difference in the time
taken to perform a "gn gen out/Release" on a full Chrome
for Android checkout.

This also needs a PRESUBMIT.py step that ensures that all .pydeps
files are up-to-date with regards to their corresponding .py source
files (this can be done with 'gen_pydeps.py --check .../foo.py')

+ Remove some --depfile options on a few Python scripts that
  don't need it anymore (e.g. when their inputs and outputs
  are now fully known by GN at parse time).

+ Remove uses of compute_inputs_for_analyze in rules.gni and
  internal_rules.gni (as well as exec_script() calls to
  print_python_deps.py). Note that the variable itself is still
  needed for grit inputs, see tools/grit/grit_rule.gni for
  details).

BUG=870845,843562
R=agrieve@chromium.org, estevenson@chromium.org

Change-Id: Id1f606a0c9df9e4e8971fd885ac0394103ca7b03
Reviewed-on: https://chromium-review.googlesource.com/1163512
Reviewed-by: agrieve <agrieve@chromium.org>
Reviewed-by: Eric Stevenson <estevenson@chromium.org>
Commit-Queue: David Turner <digit@chromium.org>
Cr-Commit-Position: refs/heads/master@{#581158}