Ignore builders without result

Ignore builders without result

The newly added "WebKit Mac10.10" builder has no result yet, and caused
crash of rebaseline script.

BUG=502141
TBR=joelo@chromium.org,dpranke@chromium.org

Committed: https://src.chromium.org/viewvc/blink?view=rev&revision=197432

[Xianzhu, 2015-06-19 05:25:29]

Committed patchset #1 (id:1) manually as 197432 (presubmit successful).

[Dirk Pranke, 2015-06-20 00:16:50]

what happens if we fail to get the results from a master we need to do the
rebaselining correctly?

[Xianzhu, 2015-06-20 00:42:28]

On 2015/06/20 00:16:50, Dirk Pranke wrote:
> what happens if we fail to get the results from a master we need to do the
> rebaselining correctly?

NetworkTransaction.run() will retry the request until a correct result or 404
(will return None) is got, or timeout (will raise exception).

[Dirk Pranke, 2015-06-20 00:59:48]

On 2015/06/20 00:42:28, Xianzhu wrote:
> On 2015/06/20 00:16:50, Dirk Pranke wrote:
> > what happens if we fail to get the results from a master we need to do the
> > rebaselining correctly?
> 
> NetworkTransaction.run() will retry the request until a correct result or 404
> (will return None) is got, or timeout (will raise exception).

That's not quite what I mean.

What I mean is, if we really need to get a result from, say, Mavericks in order
to make sure the graph
is updated correctly, if we return None here and bypass that builder, does that
mean that we'll delete
the [ NeedsRebaseline ] entries and rebaseline incorrectly?

[Xianzhu, 2015-06-20 02:57:31]

On 2015/06/20 00:59:48, Dirk Pranke wrote:
> On 2015/06/20 00:42:28, Xianzhu wrote:
> > On 2015/06/20 00:16:50, Dirk Pranke wrote:
> > > what happens if we fail to get the results from a master we need to do the
> > > rebaselining correctly?
> > 
> > NetworkTransaction.run() will retry the request until a correct result or
404
> > (will return None) is got, or timeout (will raise exception).
> 
> That's not quite what I mean.
> 
> What I mean is, if we really need to get a result from, say, Mavericks in
order
> to make sure the graph
> is updated correctly, if we return None here and bypass that builder, does
that
> mean that we'll delete
> the [ NeedsRebaseline ] entries and rebaseline incorrectly?

I'm confused: why would someone rebaseline for a builder (e.g. Mac10.10) which
doesn't exist or has not generate any result?
Another question is: when will Mac10.10 generate the first result?

The CLs were TBR'ed because I didn't want rebaseline-o-matic to be blocked too
long. I can revert them when we have Mac10.10 setup.

[joelo, 2015-06-22 10:03:54]

On 2015/06/20 02:57:31, Xianzhu wrote:
> On 2015/06/20 00:59:48, Dirk Pranke wrote:
> > On 2015/06/20 00:42:28, Xianzhu wrote:
> > > On 2015/06/20 00:16:50, Dirk Pranke wrote:
> > > > what happens if we fail to get the results from a master we need to do
the
> > > > rebaselining correctly?
> > > 
> > > NetworkTransaction.run() will retry the request until a correct result or
> 404
> > > (will return None) is got, or timeout (will raise exception).
> > 
> > That's not quite what I mean.
> > 
> > What I mean is, if we really need to get a result from, say, Mavericks in
> order
> > to make sure the graph
> > is updated correctly, if we return None here and bypass that builder, does
> that
> > mean that we'll delete
> > the [ NeedsRebaseline ] entries and rebaseline incorrectly?
> 
> I'm confused: why would someone rebaseline for a builder (e.g. Mac10.10) which
> doesn't exist or has not generate any result?
> Another question is: when will Mac10.10 generate the first result?
> 
> The CLs were TBR'ed because I didn't want rebaseline-o-matic to be blocked too
> long. I can revert them when we have Mac10.10 setup.

Ah, sorry for breaking things! I'll get the builder up asap.

It's not clear to me if this will cause auto-rebaseline to rebaseline
incorrectly if a builder for a current version is down. I'll run a test when I
get the chance. We should probably handle this properly so future versions don't
break the script when added.

[Dirk Pranke, 2015-06-22 22:59:49]

On 2015/06/22 10:03:54, joelo wrote:
> On 2015/06/20 02:57:31, Xianzhu wrote:
> > On 2015/06/20 00:59:48, Dirk Pranke wrote:
> > > On 2015/06/20 00:42:28, Xianzhu wrote:
> > > > On 2015/06/20 00:16:50, Dirk Pranke wrote:
> > > > > what happens if we fail to get the results from a master we need to do
> the
> > > > > rebaselining correctly?
> > > > 
> > > > NetworkTransaction.run() will retry the request until a correct result
or
> > 404
> > > > (will return None) is got, or timeout (will raise exception).
> > > 
> > > That's not quite what I mean.
> > > 
> > > What I mean is, if we really need to get a result from, say, Mavericks in
> > order
> > > to make sure the graph
> > > is updated correctly, if we return None here and bypass that builder, does
> > that
> > > mean that we'll delete
> > > the [ NeedsRebaseline ] entries and rebaseline incorrectly?
> > 
> > I'm confused: why would someone rebaseline for a builder (e.g. Mac10.10)
which
> > doesn't exist or has not generate any result?

My concern is the same one Joel metnions, below: what happens if a bot
that is normally functioning fine starts returning 404s? I'm less concerned
about
bots that have never worked (like 10.10 currently).

[Xianzhu, 2015-06-22 23:02:17]

On 2015/06/22 22:59:49, Dirk Pranke wrote:
> On 2015/06/22 10:03:54, joelo wrote:
> > On 2015/06/20 02:57:31, Xianzhu wrote:
> > > On 2015/06/20 00:59:48, Dirk Pranke wrote:
> > > > On 2015/06/20 00:42:28, Xianzhu wrote:
> > > > > On 2015/06/20 00:16:50, Dirk Pranke wrote:
> > > > > > what happens if we fail to get the results from a master we need to
do
> > the
> > > > > > rebaselining correctly?
> > > > > 
> > > > > NetworkTransaction.run() will retry the request until a correct result
> or
> > > 404
> > > > > (will return None) is got, or timeout (will raise exception).
> > > > 
> > > > That's not quite what I mean.
> > > > 
> > > > What I mean is, if we really need to get a result from, say, Mavericks
in
> > > order
> > > > to make sure the graph
> > > > is updated correctly, if we return None here and bypass that builder,
does
> > > that
> > > > mean that we'll delete
> > > > the [ NeedsRebaseline ] entries and rebaseline incorrectly?
> > > 
> > > I'm confused: why would someone rebaseline for a builder (e.g. Mac10.10)
> which
> > > doesn't exist or has not generate any result?
> 
> My concern is the same one Joel metnions, below: what happens if a bot
> that is normally functioning fine starts returning 404s? I'm less concerned
> about
> bots that have never worked (like 10.10 currently).

I don't know much about the servers. With this concern, I'll revert the changes.
Can I do it now without breakage?

[Dirk Pranke, 2015-06-22 23:03:50]

On 2015/06/22 23:02:17, Xianzhu wrote:
> I don't know much about the servers. With this concern, I'll revert the
changes.
> Can I do it now without breakage?

I wouldn't worry about it for now. This is probably a mostly theoretical
concern.

Let's just leave things the way they are and get 10.10 up and running and let
Joel 
worry about the long run ;).

[Xianzhu, 2015-06-22 23:05:17]

Is it possible to run rebaseline-o-matic with local changes? If yes, I'll make
local changes to workaround failures instead of submitting such changes.

[joelo, 2015-06-22 23:09:21]

Yup, you can have the script hardcoded locally to ignore 10.10 for now. But as
Dirk mentioned I wouldn't worry about it too much. I'm filing to request to
bring up the 10.10 bot right now.