| commit | 13510755ac11e6e1f58c34ef8c9bd4cf925c8d70 | [log] [tgz] |
|---|---|---|
| author | jinsukkim <jinsukkim@chromium.org> | Wed Jun 29 00:45:26 2016 |
| committer | Commit bot <commit-bot@chromium.org> | Wed Jun 29 00:46:57 2016 |
| tree | 8125e3f74185ca92ae3c0849bf2539a8b7e9ec1e | |
| parent | 2d092db5bb51f021d7213076ad3b33590be6efc4 [diff] |
Replace ICU with CED for auto encoding detection This is a drop-in replacement of ICU library performing automatic text encoding detection with CED (Compact Encdoing Detection). CED is used extensively in Google for every crawled web page, email message, query string, etc., and recently open-sourced for public use. (https://github.com/google/compact_enc_det) Also it is a much better alternative to ICU in terms of speed. ICU introduces significant regression in page loading (up to 30%): = ICU auto-detection vs. TOT = page_cycler.typical_25:cold_times.page_load_time 1085.13±9.31% 754.28±12.03% (30.49%) http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_21-20-58 while CED adds virtually no additional loading time (delta < sigma): = CED auto-detection vs. TOT = page_cycler.typical_25:cold_times.page_load_time ms 705.70±9.49% vs. 760.31±11.90% (-7.74%) http://storage.googleapis.com/chromium-telemetry/html-results/results-2016-05-08_20-37-54 With CED, it is feasible to turn on auto encoding detection by default so that web pages without encoding label can be taken care of. It will be done in a follow-up CL. BUG=597488 Review-Url: https://codereview.chromium.org/2081653007 Cr-Commit-Position: refs/heads/master@{#402622}