Commit Graph

51 Commits

Author SHA1 Message Date
Andrew Scott 736b1f9fd4 fix(compiler): recover from an incomplete open tag at the end of a file (#41054)
The compiler's parsing code has logic to recover from incomplete open
tags (i.e. `<div`) but the recovery logic does not handle when the
incomplete tag is terminated by an EOF. This commit updates the logic to
allow for the EOF character to be interpreted as the end of the tag open
so that the parser can continue processing. It will then fail to find
the end tag and recover by marking the open tag as incomplete.

Part of https://github.com/angular/vscode-ng-language-service/issues/1140

PR Close #41054
2021-03-03 09:58:56 -08:00
JoostK c18c7e23ec fix(compiler): exclude trailing whitespace from element source spans (#40513)
If the template parse option `leadingTriviaChars` is configured to
consider whitespace as trivia, any trailing whitespace of an element
would be considered as leading trivia of the subsequent element, such
that its `start` span would start _after_ the whitespace. This means
that the start span cannot be used to mark the end of the current
element, as its trailing whitespace would then be included in its span.
Instead, the full start of the subsequent element should be used.

To harden the tests that for the Ivy parser, the test utility `parseR3`
has been adjusted to use the same configuration for `leadingTriviaChars`
as would be the case in its production counterpart `parseTemplate`. This
uncovered another bug in offset handling of the interpolation parser,
where the absolute offset was computed from the start source span
(which excludes leading trivia) whereas the interpolation expression
would include the leading trivia. As such, the absolute offset now also
uses the full start span.

Fixes #39148

PR Close #40513
2021-01-28 08:53:02 -08:00
Kristiyan Kostadinov 66c27ffdfc fix(compiler): incorrectly inferring content type of SVG-specific title tag (#40259)
The parser has a list of tag definitions that it uses when parsing the template. Each tag has a
`contentType` which tells the parser what kind of content the tag should contain. The problem is
that the browser has two separate `title` tags (`HTMLTitleElement` and `SVGTitleElement`) and each
of them has to have a different `contentType`, otherwise the parser will throw an error further down
the pipeline.

These changes update the tag definitions so that each tag name can have multiple content types
associated with it and the correct one can be returned based on the element's prefix.

Fixes #31503.

PR Close #40259
2021-01-11 15:35:23 -08:00
Marcono1234 3e1e5a15ba docs: update links to use HTTPS as protocol (#39718)
PR Close #39718
2020-11-20 12:52:16 -08:00
Pete Bacon Darwin 43d8e9aad2 refactor(compiler): capture `fullStart` locations when tokenizing (#39486)
This commit ensures that when leading whitespace is skipped by
the tokenizer, the original start location (before skipping) is captured
in the `fullStart` property of the token's source-span.

PR Close #39486
2020-11-06 09:01:37 -08:00
Ayaz Hafiz 6ae3b68acf feat(compiler): Parse and recover on incomplete opening HTML tags (#38681)
Let's say we have a code like

```html
<div<span>123</span>
```

Currently this gets parsed into a tree with the element tag `div<span`.
This has at least two downsides:

- An incorrect diagnostic that `</span>` doesn't close an element is
  emitted.
- A consumer of the parse tree using it for editor services is unable to
  provide correct completions for the opening `<span>` tag.

This patch attempts to fix both issues by instead parsing the code into
the same tree that would be parsed for `<div></div><span>123</span>`.

In particular, we do this by optimistically scanning an open tag as
usual, but if we do not notice a terminating '>', we mark the tag as
"incomplete". A parser then emits an error for the incomplete tag and
adds a synthetic (recovered) element node to the tree with the
incomplete open tag's name.

What's the downside of this? For one, a breaking change.

<ol>
<li>

The first breaking change is that `<` symbols that are ambiguously text
or opening tags will be parsed as opening tags instead of text in
element bodies. Take the code

```html
<p>a<b</p>
```

Clearly we cannot have the best of both worlds, and this patch chooses
to swap the parsing strategy to support the new feature. Of course, `<`
can still be inserted as text via the `&lt;` entity.

</li>
</ol>

Part of #38596

PR Close #38681
2020-09-21 12:27:01 -07:00
Pete Bacon Darwin 1d8c5d88cd refactor(compiler): `element.sourceSpan` should span the `outerHTML` (#38581)
Previously, the `sourceSpan` and `startSourceSpan` were the same
object, which meant that you had the following situation:

```
element = <div>some content</div>
sourceSpan = <div>
startSourceSpan = <div>
endSourceSpan = </div>
```

This made `sourceSpan` redundant and meant that if you
wanted a span for the whole element including its content
and closing tag, it had to be computed.

Now `sourceSpan` is separated from `startSourceSpan`
resulting in:

```
element = <div>some content</div>
sourceSpan = <div>some content</div>
startSourceSpan = <div>
endSourceSpan = </div>
```

PR Close #38581
2020-09-02 14:47:31 -07:00
Pete Bacon Darwin a68f1a78a7 refactor(compiler): element.startSourceSpan is required (#38581)
Previously, the `startSourceSpan` property could be null
but in reality it is always well defined - except for a legacy
case in the old i18n extraction/merging code, where the
typings for source-spans are already being undermined.

Making this property non-null, simplifies code elsewhere
in the project.

PR Close #38581
2020-09-02 14:47:28 -07:00
Pete Bacon Darwin 86e11f1110 refactor(compiler): move the line-ending handling decision (#38581)
Previously the lexer was responsible for deciding whether an "inline"
template should also have its line-endings normalized.

Now this decision is made higher up in the call stack to allow more
flexibility in the parser/lexer.

PR Close #38581
2020-09-02 14:47:25 -07:00
Joey Perrott ff9f4de4f1 fix(compiler): update unparsable character reference entity error messages (#38319)
Within an angular template, when a character entity is unable to be parsed, previously a generic
unexpected character error was thrown.  This does not properly express the issue that was discovered
as the issue is actually caused by the discovered character making the whole of the entity unparsable.
The compiler will now instead inform via the error message what string was attempted to be parsed
and what it was attempted to be parsed as.

Example, for this template:
```
<p>
  &#x123p
</p>
```
Before this change:
`Unexpected character "p"`

After this change:
`Unable to parse entity "&#x123p" - hexadecimal character reference entities must end with ";"`

Fixes #26067

PR Close #38319
2020-07-31 15:32:53 -07:00
JoostK 1a7a7360b0 fix(compiler): properly associate source spans for implicitly closed elements (#38126)
HTML is very lenient when it comes to closing elements, so Angular's parser has
rules that specify which elements are implicitly closed when closing a tag.
The parser keeps track of the nesting of tag names using a stack and parsing
a closing tag will pop as many elements off the stack as possible, provided
that the elements can be implicitly closed.

For example, consider the following templates:

- `<div><br></div>`, the `<br>` is implicitly closed when parsing `</div>`,
  because `<br>` is a void element.
- `<div><p></div>`, the `<p>` is implicitly closed when parsing `</div>`,
  as `<p>` is allowed to be closed by the closing of its parent element.
- `<ul><li>A <li>B</ul>`, the first `<li>` is implicitly closed when parsing
  the second `<li>`, whereas the second `<li>` would be implicitly closed when
  parsing the `</ul>`.

In all the cases above the parsed structure would be correct, however the source
span of the closing `</div>` would incorrectly be assigned to the element that
is implicitly closed. The problem was that closing an element would associate
the source span with the element at the top of the stack, however this may not
be the element that is actually being closed if some elements would be
implicitly closed.

This commit fixes the issue by assigning the end source span with the element
on the stack that is actually being closed. Any implicitly closed elements that
are popped off the stack will not be assigned an end source span, as the
implicit closing implies that no ending element is present.

Note that there is a difference between self-closed elements such as `<input/>`
and implicitly closed elements such as `<input>`. The former does have an end
source span (identical to its start source span) whereas the latter does not.

Fixes #36118
Resolves FW-2004

PR Close #38126
2020-07-20 10:02:06 -07:00
Joey Perrott d1ea1f4c7f build: update license headers to reference Google LLC (#37205)
Update the license headers throughout the repository to reference Google LLC
rather than Google Inc, for the required license headers.

PR Close #37205
2020-05-26 14:26:58 -04:00
Pete Bacon Darwin 70dd27ffd8 fix(compiler): normalize line endings in ICU expansions (#36741)
The html parser already normalizes line endings (converting `\r\n` to `\n`)
for most text in templates but it was missing the expressions of ICU expansions.

In ViewEngine backticked literal strings, used to define inline templates,
were already normalized by the TypeScript parser.
In Ivy we are parsing the raw text of the source file directly so the line
endings need to be manually normalized.

This change ensures that inline templates have the line endings of ICU
expression normalized correctly, which matches the ViewEngine.

In ViewEngine external templates, defined in HTML files, the behavior was
different, since TypeScript was not normalizing the line endings.
Specifically, ICU expansion "expressions" are not being normalized.
This is a problem because it means that i18n message ids can be different on
different machines that are setup with different line ending handling,
or if the developer moves a template from inline to external or vice versa.

The goal is always to normalize line endings, whether inline or external.
But this would be a breaking change since it would change i18n message ids
that have been previously computed. Therefore this commit aligns the ivy
template parsing to have the same "buggy" behavior for external templates.

There is now a compiler option `i18nNormalizeLineEndingsInICUs`, which
if set to `true` will ensure the correct non-buggy behavior. For the time
being this option defaults to `false` to ensure backward compatibility while
allowing opt-in to the desired behavior. This option's default will be
flipped in a future breaking change release.

Further, when this option is set to `false`, any ICU expression tokens,
which have not been normalized, are added to the `ParseResult` from the
`HtmlParser.parse()` method. In the future, this collection of tokens could
be used to diagnose and encourage developers to migrate their i18n message
ids. See FW-2106.

Closes #36725

PR Close #36741
2020-04-28 12:22:40 -07:00
Alex Rickabaugh 83a9159063 style(compiler): reformat of codebase with new clang-format version (#36520)
This commit reformats the packages/compiler tree using the new version of
clang-format.

PR Close #36520
2020-04-08 14:51:08 -07:00
Sonu Kapoor fced8ee40e fix(localize): allow ICU expansion case to start with any character except `}` (#36123)
Previously, an expansion case could only start with an alpha numeric character.
This commit fixes this by allowing an expansion case to start with any character
except `}`.

The [ICU spec](http://userguide.icu-project.org/formatparse/messages) is pretty vague:

> Use a "select" argument to select sub-messages via a fixed set of keywords.

It does not specify what can be a "keyword" but from looking at the surrounding syntax it
appears that it can indeed be any string that does not contain a `}` character.

Closes #31586

PR Close #36123
2020-03-23 11:37:12 -07:00
Greg Magolan a28c02bf89 build: derive ts_library dep from jasmine_node_test boostrap label if it ends in `_es5` (#34736)
PR Close #34736
2020-01-15 14:58:07 -05:00
Greg Magolan aee67f08d9 test: handle bootstrap templated_args in jasmine_node_test defaults.bzl (#34736)
PR Close #34736
2020-01-15 14:58:07 -05:00
Greg Magolan dcff76e8b9 refactor: handle breaking changes in rules_nodejs 1.0.0 (#34736)
The major one that affects the angular repo is the removal of the bootstrap attribute in nodejs_binary, nodejs_test and jasmine_node_test in favor of using templated_args --node_options=--require=/path/to/script. The side-effect of this is that the bootstrap script does not get the require.resolve patches with explicitly loading the targets _loader.js file.

PR Close #34736
2020-01-15 14:58:07 -05:00
atscott 538d0446b5 Revert "refactor: handle breaking changes in rules_nodejs 1.0.0 (#34589)" (#34730)
This reverts commit 9bb349e1c8.

PR Close #34730
2020-01-10 14:12:15 -08:00
atscott 5e60215470 Revert "test: handle bootstrap templated_args in jasmine_node_test defaults.bzl (#34589)" (#34730)
This reverts commit da4782e67f.

PR Close #34730
2020-01-10 14:12:15 -08:00
atscott 24679d8676 Revert "build: derive ts_library dep from jasmine_node_test boostrap label if it ends in `_es5` (#34589)" (#34730)
This reverts commit 79a0d007b4.

PR Close #34730
2020-01-10 14:12:14 -08:00
Greg Magolan 79a0d007b4 build: derive ts_library dep from jasmine_node_test boostrap label if it ends in `_es5` (#34589)
PR Close #34589
2020-01-10 08:32:00 -08:00
Greg Magolan da4782e67f test: handle bootstrap templated_args in jasmine_node_test defaults.bzl (#34589)
PR Close #34589
2020-01-10 08:31:59 -08:00
Greg Magolan 9bb349e1c8 refactor: handle breaking changes in rules_nodejs 1.0.0 (#34589)
The major one that affects the angular repo is the removal of the bootstrap attribute in nodejs_binary, nodejs_test and jasmine_node_test in favor of using templated_args --node_options=--require=/path/to/script. The side-effect of this is that the bootstrap script does not get the require.resolve patches with explicitly loading the targets _loader.js file.

PR Close #34589
2020-01-10 08:31:59 -08:00
Greg Magolan 9a68f23dd2 build: ts_web_test & ts_web_test_suite deprecated in favor of karma_web_test & karma_web_test_suite (#33802)
This is a breaking change in nodejs rules 0.40.0 as part of the API review & cleanup for the 1.0 release. Their APIs are identical as ts_web_test was just karma_web_test without the config_file attribute.

PR Close #33802
2019-11-13 13:33:38 -08:00
Pete Bacon Darwin 0ddf0c4895 fix(compiler): do not remove whitespace wrapping i18n expansions (#31962)
Similar to interpolation, we do not want to completely remove whitespace
nodes that are siblings of an expansion.

For example, the following template

```html
<div>
  <strong>items left<strong> {count, plural, =1 {item} other {items}}
</div>
```

was being collapsed to

```html
<div><strong>items left<strong>{count, plural, =1 {item} other {items}}</div>
```

which results in the text looking like

```
items left4
```

instead it should be collapsed to

```html
<div><strong>items left<strong> {count, plural, =1 {item} other {items}}</div>
```

which results in the text looking like

```
items left 4
```

---

**Analysis of the code and manual testing has shown that this does not cause
the generated ids to change, so there is no breaking change here.**

PR Close #31962
2019-08-09 12:03:50 -07:00
Pete Bacon Darwin 304a12f027 feat(compiler): support skipping leading trivia in template source-maps (#30095)
Leading trivia, such as whitespace or comments, is
confusing for developers looking at source-mapped
templates, since they expect the source-map segment
to start after the trivia.

This commit adds skipping trivial characters to the lexer;
and then implements that in the template parser.

PR Close #30095
2019-04-25 12:36:54 -07:00
Pawel Kozlowski dafbbf8b64 fix(core): parse incorrect ML open tag as text (#29328)
This PR alligns markup language lexer with the previous behaviour in version 7.x:
https://stackblitz.com/edit/angular-iancj2

While this behaviour is not perfect (we should be giving users an error message
here about invalid HTML instead of assuming text node) this is probably best we
can do without more substential re-write of lexing / parsing infrastructure.

This PR just fixes #29231 and restores VE behaviour - a more elaborate fix will
be done in a separate PR as it requries non-trivial rewrites.

PR Close #29328
2019-03-19 23:23:31 -04:00
Matias Niemelä a3ec058f6b revert: fix(core): parse incorrect ML open tag as text (#29328) 2019-03-19 11:12:32 -07:00
Pawel Kozlowski 4605df83e1 fix(core): parse incorrect ML open tag as text (#29328)
This PR alligns markup language lexer with the previous behaviour in version 7.x:
https://stackblitz.com/edit/angular-iancj2

While this behaviour is not perfect (we should be giving users an error message
here about invalid HTML instead of assuming text node) this is probably best we
can do without more substential re-write of lexing / parsing infrastructure.

This PR just fixes #29231 and restores VE behaviour - a more elaborate fix will
be done in a separate PR as it requries non-trivial rewrites.

PR Close #29328
2019-03-19 13:30:20 -04:00
Pawel Kozlowski f2dc32e5c7 fix(core): don't wrap `<tr>` and `<col>` elements into a required parent (#29219)
BREAKING CHANGE:

Certain elements (like `<tr>` or `<col>`) require parent elements to be of a certain type by the HTML specification
(ex. <tr> can only be inside <tbody> / <thead>). Before this change Angular template parser was auto-correcting
"invalid" HTML using the following rules:
- `<tr>` would be wrapped in `<tbody>` if not inside `<tbody>`, `<tfoot>` or `<thead>`;
- `<col>` would be wrapped in `<colgroup>` if not inside `<colgroup>`.

This meachanism of automatic wrapping / auto-correcting was problematic for several reasons:
- it is non-obvious and arbitrary (ex. there are more HTML elements that has rules for parent type);
- it is incorrect for cases where `<tr>` / `<col>` are at the root of a component's content, ex.:

```html
<projecting-tr-inside-tbody>
  <tr>...</tr>
</projecting-tr-inside-tbody>
```

In the above example the `<projecting-tr-inside-tbody>` component culd be "surprised" to see additional
`<tbody>` elements inserted by Angular HTML parser.

PR Close #29219
2019-03-14 03:07:01 -04:00
Pete Bacon Darwin f7c867ebc2 fix(ivy): correctly tokenize escaped characters in templates (#28978)
Previously the start of a character indicated by an escape sequence
was being incorrectly computed by the lexer, which caused tokens
to include the start of the escaped character sequence in the
preceding token. In particular this affected the name extracted
from opening tags if the name was terminated by an escape sequence.
For example, `<t\n>` would have the name `t\` rather than `t`.

This fix refactors the lexer to use a "cursor" object to iterate over
the characters in the template source. There are two cursor implementations,
one expects a simple string, the other expects a string that contains
JavaScript escape sequences that need to be unescaped.

PR Close #28978
2019-02-28 02:44:19 -08:00
Pete Bacon Darwin 76979e12c9 refactor(compiler): remove not-null operator hack from lexer (#28978)
The parts of a token are supposed to be an array of not-null strings,
but we were using `null` for tags that had no prefix. This has been
fixed to use the empty string in such cases, which allows the `null !`
hack to be removed.

PR Close #28978
2019-02-28 02:44:19 -08:00
Pete Bacon Darwin c0dac184cd fix(compiler): markup lexer should not capture quotes in attribute value (#28055)
When tokenizing markup (e.g. HTML) element attributes
can have quoted or unquoted values (e.g. `a=b` or `a="b"`).
The `ATTR_VALUE` tokens were capturing the quotes, which
was inconsistent and also affected source-mapping.

Now the tokenizer captures additional `ATTR_QUOTE` tokens,
which the HTML related parsers understand and factor into their
token parsing.

PR Close #28055
2019-02-12 20:58:27 -08:00
Pete Bacon Darwin 2424184d42 feat(compiler): support tokenizing escaped strings (#28055)
In order to support source mapping of templates, we need
to be able to tokenize the template in its original context.
When the template is defined inline as a JavaScript string
in a TS/JS source file, the tokenizer must be able to handle
string escape sequences, such as `\n` and `\"` as they
appear in the original source file.

This commit teaches the lexer how to unescape these
sequences, but only when the `escapedString` option is
set to true.  Otherwise there is no change to the tokenizing
behaviour.

PR Close #28055
2019-02-12 20:58:27 -08:00
Pete Bacon Darwin eeb560ac88 feat(compiler): support tokenizing a sub-section of an input string (#28055)
The lexer that does the tokenizing can now process only a part the source
string, by passing a `range` property in the `options` argument. The
locations of the nodes that are tokenized will now take into account the
position of the span in the context of the original source string.

This `range` option is, in turn, exposed from the template parser as well.

Being able to process parts of files helps to enable SourceMap support
when compiling inline component templates.

PR Close #28055
2019-02-12 20:58:27 -08:00
Pete Bacon Darwin 673ac2945c refactor(compiler): use `options` argument for parsers (#28055)
This commit consolidates the options that can modify the
parsing of text (e.g. HTML, Angular templates, CSS, i18n)
into an AST for further processing into a single `options`
hash.

This makes the code cleaner and more readable, but also
enables us to support further options to parsing without
triggering wide ranging changes to code that should not
be affected by these new options.  Specifically, it will let
us pass information about the placement of a template
that is being parsed in its containing file, which is essential
for accurate SourceMap processing.

PR Close #28055
2019-02-12 20:58:27 -08:00
Alex Rickabaugh 4effb89ae8 test(ivy): update //packages/compiler tests for Ivy (#27301)
Most of the specs in these tests are not relevant to Ivy:

//packages/compiler/test:test
//packages/compiler/test:test_web_chromium-local

However, a few test pieces of the compiler infrastructure that are used in
Ivy, and new BUILD.bazel files are created to separate them from the above
disabled targets:

//packages/compiler/test/css_parser:css_parser
//packages/compiler/test/css_parser:css_parser_web
//packages/compiler/test/expression_parser:expression_parser
//packages/compiler/test/expression_parser:expression_parser_web
//packages/compiler/test/ml_parser:ml_parser
//packages/compiler/test/ml_parser:ml_parser_web
//packages/compiler/test/selector:selector
//packages/compiler/test/selector:selector_web

PR Close #27301
2018-11-29 21:31:35 -08:00
Victor Berchet 0ebd577db4 refactor(compiler): Drop support for the deprecated `<template>`. Use `<ng-template>` instead (#22783)
BREAKING CHANGE:

The `<template>` tag was deprecated in Angular v4 to avoid collisions (i.e. when
using Web Components).

This commit removes support for `<template>`. `<ng-template>` should be used
instead.

BEFORE:

    <!-- html template -->
    <template>some template content</template>

    # tsconfig.json
    {
      # ...
      "angularCompilerOptions": {
        # ...
        # This option is no more supported and will have no effect
        "enableLegacyTemplate": [true|false]
      }
    }

AFTER:

    <!-- html template -->
    <ng-template>some template content</ng-template>

PR Close #22783
2018-03-15 14:52:22 -07:00
Victor Berchet cb5090cdc8 fix(compiler): fix ICU select messages to use male/female/other (#21713)
related to #21694

PR Close #21713
2018-01-23 16:32:23 -08:00
Misko Hevery 47e251a80a build: remove `main()` from specs (#21053)
PR Close #21053
2017-12-22 13:10:51 -08:00
Pawel Kozlowski 13613d4acb fix(compiler): skip &nbsp; when trimming / removing whitespaces (#19310)
Fixes #19304
2017-09-25 13:33:31 -07:00
Olivier Combe 0f5c70d563 build: update npm dependencies (#19328)
PR Close #19328
2017-09-22 13:20:52 -07:00
Pawel Kozlowski d2c0d986d4 perf(core): add option to remove blank text nodes from compiled templates 2017-08-14 13:26:16 -07:00
Victor Berchet 381471d338 fix(compiler): fix for element needing implicit parent placed in top-level ng-container
fixes #18314
2017-07-31 11:30:19 -07:00
Olivier Combe a3a54299af fix(compiler): allow numbers for ICU message cases in lexer (#18095)
Closes #18095
Fixes #17799
2017-07-20 17:41:54 -05:00
Victor Berchet c723d42d0a refactor: fix typos (#18000) 2017-07-07 16:55:17 -07:00
Trotyl Yu 9315ab88d7 test(compiler): fix typo "mamespace" 2017-06-16 11:17:15 -07:00
Miško Hevery 09d9f5fe54 fix(compiler): Update types for TypeScript nullability support 2017-04-10 15:26:33 -06:00
Dzmitry Shylovich 5f9fb911f7 fix(compiler): improve error msg for unexpected closing tags (#14747)
Closes #6652
2017-03-10 13:05:17 -08:00