honeymoose
/
angular-cn
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(ngcc): do not spawn more processes than intended in parallel mode (#36280) 

When running in parallel mode, ngcc spawns multiple worker processed to
process the various entry-points. The number of max allowed processes is
determined by the number of CPU cores available to the OS. There is also
currently an [upper limit of 8][1]. The number of active workers is in
turn inferred by the number of [task assignments][2].

In the past, counting the entries of `ClusterMaster#taskAssignments` was
enough, because worker processes were spawned eagerly at the beginning
and corresponding entries were created in `taskAssignments`.
Since #35719 however, worker processes are spawned lazily on an as
needed basis. Because there is some delay between
[spawning a process][3] and [inserting it][4] into `taskAssignments`,
there is a short period of time when `taskAssignment.size` does not
actually represent the number of spawned processes. This can result in
spawning more than `ClusterMaster#workerCount` processes.

An example of this can be seen in #36278, where the debug logs indicate
9 worker processes had been spawned (`All 9 workers are currently busy`)
despite the hard limit of 8.

This commit fixes this by using `cluster.workers` to compute the number
of spawned worker processes. `cluster.workers` is updated synchronously
with `cluster.fork()` and thus reflects the number of spawned workers
accurately at all times.

[1]: https://github.com/angular/angular/blob/b8e9a30d3b6/packages/compiler-cli/ngcc/src/main.ts#L429
[2]: https://github.com/angular/angular/blob/b8e9a30d3b6/packages/compiler-cli/ngcc/src/execution/cluster/master.ts#L108
[3]: https://github.com/angular/angular/blob/b8e9a30d3b6/packages/compiler-cli/ngcc/src/execution/cluster/master.ts#L110
[4]: https://github.com/angular/angular/blob/b8e9a30d3b6/packages/compiler-cli/ngcc/src/execution/cluster/master.ts#L199

PR Close #36280
This commit is contained in:
George Kalpakas 2020-03-27 19:30:39 +02:00 committed by Alex Rickabaugh
parent 2d0847c307
commit 5cee709266
1 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
packages/compiler-cli/ngcc/src/execution/cluster/master.ts
View File

@ -105,14 +105,14 @@ export class ClusterMaster {
} }
if (!isWorkerAvailable) { if (!isWorkerAvailable) {
if (this.taskAssignments.size < this.workerCount) { const spawnedWorkerCount = Object.keys(cluster.workers).length;
if (spawnedWorkerCount < this.workerCount) {
this.logger.debug('Spawning another worker process as there is more work to be done.'); this.logger.debug('Spawning another worker process as there is more work to be done.');
cluster.fork(); cluster.fork();
} else { } else {
// If there are no available workers or no available tasks, log (for debugging purposes). // If there are no available workers or no available tasks, log (for debugging purposes).
this.logger.debug( this.logger.debug(
`All ${this.taskAssignments.size} workers are currently busy and cannot take on more ` + `All ${spawnedWorkerCount} workers are currently busy and cannot take on more work.`);
'work.');
} }
} else { } else {
const busyWorkers = Array.from(this.taskAssignments) const busyWorkers = Array.from(this.taskAssignments)