commit 3035f7c165e19a7425d53f3e41b60b9a5ce6dc4e
author Robert Iannucci <iannucci@chromium.org> Wed Mar 13 21:09:30 2019
committer Commit Bot <commit-bot@chromium.org> Wed Mar 13 21:09:30 2019
tree aa7299fb1d44623562a5b568af362bf42c091193
parent aef5bb3df38a8c316b2af0301be808e34c749583

Refactor step names.

Previously in recipe engine steps were identified by a ragtag group of
three attributes:
  * base_name: a '.' concatenated series of namespaces
  * name: the last bit of base_name
  * nest_level: A weird integer which sort-of counted how many '.'s were in
    base_name. ish.

Additionally, the context of 'how nested' we were was passed around
internally as a single 'name_prefix' string, plus the nest_level. It
was possible to modify name_prefix without also using nest_level, but
you could add '.' to name_prefix this way and it was a big mess.

Because build.proto represents steps as '|' separated series of strings
(and reserves the '|' character for this purpose), we need to push this
abstraction down to the StreamEngine level (i.e. what converts recipes'
internal state to @@@annotation@@@ or build.proto).

This CL:

Replaces all the previous name state with `namespace`; A tuple of string bits
(none of which contain "|"). The last hunk of the namespace is typically '', but
can be added to with name_prefix.

Replaces all internal representations of a step's `name` (and base_name and
nest_level) with `name_tokens`: as a tuple of string bits (none of which contain
"|").

For backwards compatibility with existing user code (whose tests explicitly
include strings joined with ".") this CL keeps the test-case step name
representation as `'.'.join(s)`. We'll change these in a followup CL so that:
  * Users identify step names in their tests by their tuple representation.
  * The test expectation format records name as a list of string bits.

This CL also fixes three long-standing bugs:
  * api.step.nest and api.context are now fully consistent with each other.
    api.step.nest is still preferred (because then you get a manipulable step)
    but at least the state won't be hosed if you use api.context.
  * Step name deduplication now knows about namespaces. Now if you ran:
      Nest
        Step 1
        Cool
      Nest
        Step 2
        Cool

    It will correctly result in:
      Nest
      Nest|Step 1
      Nest|Cool
      Nest (2)
      Nest (2)|Step 2
      Nest (2)|Cool

    Instead of the illegal:
      Nest
      Nest|Step 1
      Nest|Cool
      Nest
      Nest|Step 2
      Nest|Cool (2)
  * Previously, because 'nest' and 'context' didn't agree with each other
    it was possible to tweak 'name_prefix' independently of 'nest_level'
    with unpredictable results. Now 'name_prefix' and 'namespace' have a
    fixed relationship: name_prefix applies a string prefix to any steps
    and new namespaces created. When creating a new namespace, name_prefix
    resets to ''. Internally they are stored together as `context._namespace`
    which has the form of `(some, namespace, strings, name_prefix)`, with
    this initially set to `('',)`.

R=martiniss@chromium.org

Recipe-Nontrivial-Roll: infra
Recipe-Nontrivial-Roll: build
Recipe-Nontrivial-Roll: build_limited_scripts_slave
Recipe-Nontrivial-Roll: release_scripts
Bug: 909848
Change-Id: I37c57c04a324206f16f69d3660f0784fcb3e3be3
Reviewed-on: https://chromium-review.googlesource.com/c/infra/luci/recipes-py/+/1512355
Commit-Queue: Robbie Iannucci <iannucci@chromium.org>
Reviewed-by: Stephen Martinis <martiniss@chromium.org>