Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up CI with Azure Pipelines #6495

Merged
merged 38 commits into from
Nov 14, 2018
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
b631120
Set up CI with Azure Pipelines
arcanis Oct 4, 2018
7f05fe3
Update azure-pipelines.yml
arcanis Oct 4, 2018
a1f0994
Update azure-pipelines.yml
arcanis Oct 5, 2018
573c9e7
Update package.json
arcanis Oct 5, 2018
0d801d0
Adds debug
Oct 5, 2018
3e168d7
Tries using bash
Oct 5, 2018
808eb7e
Tweaks timeouts
Oct 5, 2018
95eb3d7
Fixes various things related to the win32 compat
arcanis Oct 19, 2018
b95bbc0
I get the same 22 failed tests with/without the delay.
Nov 5, 2018
ea5810e
Fix makeFakeBinary for win32.
Nov 5, 2018
6830dd3
Check custom output instead of versions.
Nov 5, 2018
3a0c614
Remove quotes from echo (not removed in Windows).
Nov 5, 2018
c743366
Some tweaks
Nov 5, 2018
2c6949f
Fixes syntax
Nov 5, 2018
2312b35
Merge remote-tracking branch 'upstream/master' into azure-pipelines
Nov 5, 2018
596e8c1
Fixes tests on Windows
Nov 5, 2018
7c5a6be
Adds test reporting
Nov 5, 2018
c5dfc47
Fixes things (?)
Nov 5, 2018
9273271
Fixes indent
Nov 5, 2018
0c17392
Always run the test result publishing
Nov 5, 2018
420f74f
Fixes yml
Nov 5, 2018
6b2b594
Use node scripts to avoid Windows interoperability issues.
Nov 7, 2018
b7cf111
Fixes the "must use the same Node as Yarn itself" test
Nov 7, 2018
15bc762
Fixes escaping from within the tests
arcanis Nov 8, 2018
fbe8ced
Removes the accidental .only flag
arcanis Nov 8, 2018
0c4bd14
Fixes a test on Windows
arcanis Nov 8, 2018
3c1a97e
Add support for different drives in Windows.
Nov 8, 2018
990f43a
No need to getPackageLocation
Nov 9, 2018
9cf85d6
Add support for environment and arguments by using a custom solution …
Nov 9, 2018
1bbee27
Merge remote-tracking branch 'origin/master' into azure-pipelines
Nov 9, 2018
1d9ea16
Missing escape on path in pnpapi test.
Nov 9, 2018
adf5a2b
Some cleaning
arcanis Nov 12, 2018
a2dabc9
Runs tests on all three platforms
arcanis Nov 12, 2018
e786a9b
Fixes write paths
arcanis Nov 12, 2018
f96f54f
Fixes normalizePath
arcanis Nov 12, 2018
8f72462
Disables pkg-tests from CircleCI (moved to Azure)
arcanis Nov 12, 2018
3a8e9e5
Increase Windows timeouts and better name Test results (#6680)
pablonete Nov 14, 2018
649ae72
Revert "Disables pkg-tests from CircleCI (moved to Azure)"
arcanis Nov 14, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 107 additions & 0 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Node.js
# Build a general Node.js project with npm.
# Add steps that analyze code, save build artifacts, deploy, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/javascript

jobs:

- job: Windows

pool: 'Hosted VS2017'

strategy:
matrix:
# node_8_x:
# node_version: 8.x
node_10_x:
node_version: 10.x

steps:
- task: NodeTool@0
inputs:
versionSpec: $(node_version)
displayName: 'Install Node.js'

- bash: |
yarn
yarn run build
ls -lhR bin
ls -lhR lib
displayName: 'install and build'

- script: |
cd packages/pkg-tests
yarn
displayName: 'install the acceptance tests'

- script: |
cd packages/pkg-tests
yarn jest yarn --detectOpenHandles
displayName: 'run the acceptance tests'

# - job: Linux
#
# pool:
# vmImage: 'Ubuntu 16.04'
#
# strategy:
# matrix:
# node_8_x:
# node_version: 8.x
# node_10_x:
# node_version: 10.x
#
# steps:
# - task: NodeTool@0
# inputs:
# versionSpec: $(node_version)
# displayName: 'Install Node.js'
#
# - script: |
# yarn
# yarn build
# displayName: 'install and build'
#
# - script: |
# cd packages/pkg-tests
# yarn
# displayName: 'install the acceptance tests'
#
# - script: |
# cd packages/pkg-tests
# yarn jest yarn
# displayName: 'run the acceptance tests'
#
# - job: OSX
#
# pool:
# vmImage: 'macOS 10.13'
#
# strategy:
# matrix:
# node_8_x:
# node_version: 8.x
# node_10_x:
# node_version: 10.x
#
# steps:
# - task: NodeTool@0
# inputs:
# versionSpec: $(node_version)
# displayName: 'Install Node.js'
#
# - script: |
# yarn
# yarn build
# displayName: 'install and build'
#
# - script: |
# cd packages/pkg-tests
# yarn
# displayName: 'install the acceptance tests'
#
# - script: |
# cd packages/pkg-tests
# yarn jest yarn
# displayName: 'run the acceptance tests'
#
2 changes: 1 addition & 1 deletion packages/pkg-tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
},
"jest": {
"modulePathIgnorePatterns": [
"<rootDir>[/\\\\]pkg-tests-fixtures[/\\\\]packages[/\\\\]"
"<rootDir>/pkg-tests-fixtures/packages/"
]
}
}
2 changes: 2 additions & 0 deletions packages/pkg-tests/pkg-tests-core/sources/utils/exec.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ exports.execFile = function(
if (error) {
reject(error);
} else {
stdout = stdout.replace(/\r\n?/g, `\n`);
stderr = stderr.replace(/\r\n?/g, `\n`);
resolve({stdout, stderr});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️ no need for explicit template string use above.

}
});
Expand Down
8 changes: 4 additions & 4 deletions packages/pkg-tests/pkg-tests-core/sources/utils/fs.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,13 @@ exports.packToStream = function packToStream(
const packStream = tarFs.pack(source, {
map: header => {
if (true) {
header.name = path.resolve('/', header.name);
header.name = path.relative('/', header.name);
header.name = path.posix.resolve('/', header.name);
header.name = path.posix.relative('/', header.name);
}

if (virtualPath) {
header.name = path.resolve('/', virtualPath, header.name);
header.name = path.relative('/', header.name);
header.name = path.posix.resolve('/', virtualPath, header.name);
header.name = path.posix.relative('/', header.name);
}

return header;
Expand Down
6 changes: 6 additions & 0 deletions packages/pkg-tests/pkg-tests-core/sources/utils/tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -364,3 +364,9 @@ exports.generatePkgDriver = function generatePkgDriver({runDriver}: {|runDriver:

return withConfig({});
};

exports.testIf = function testIf(condition, ...args) {
if (condition()) {
test(...args);
}
};
11 changes: 7 additions & 4 deletions packages/pkg-tests/pkg-tests-specs/sources/pnp.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ const {relative, isAbsolute} = require('path');

const {
fs: {createTemporaryFolder, readFile, readJson, writeFile, writeJson},
tests: {getPackageDirectoryPath},
tests: {getPackageDirectoryPath, testIf},
} = require('pkg-tests-core');

module.exports = makeTemporaryEnv => {
Expand Down Expand Up @@ -619,7 +619,8 @@ module.exports = makeTemporaryEnv => {
),
);

test(
testIf(
() => process.platform !== 'win32',
`it should generate a file that can be used as an executable to resolve a request (valid request)`,
makeTemporaryEnv(
{
Expand Down Expand Up @@ -648,7 +649,8 @@ module.exports = makeTemporaryEnv => {
),
);

test(
testIf(
() => process.platform !== `win32`,
`it should generate a file that can be used as an executable to resolve a request (builtin request)`,
makeTemporaryEnv(
{
Expand All @@ -672,7 +674,8 @@ module.exports = makeTemporaryEnv => {
),
);

test(
testIf(
() => process.platform !== `win32`,
`it should generate a file that can be used as an executable to resolve a request (invalid request)`,
makeTemporaryEnv(
{
Expand Down
7 changes: 6 additions & 1 deletion packages/pkg-tests/yarn.test.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/* @flow */

const {delimiter} = require(`path`);
const {delimiter, join} = require(`path`);

const {
tests: {generatePkgDriver, startPackageServer, getPackageRegistry},
Expand Down Expand Up @@ -64,6 +64,11 @@ const pkgDriver = generatePkgDriver({
},
});

if (process.platform === `win32`) {
// Windows takes much longer to run those tests, for some reason
jest.setTimeout(15000);
}

beforeEach(async () => {
await startPackageServer();
await getPackageRegistry();
Expand Down
7 changes: 6 additions & 1 deletion src/fetchers/tarball-fetcher.js
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ export default class TarballFetcher extends BaseFetcher {
reject(new MessageError(this.config.reporter.lang('errorExtractingTarball', err.message, tarballPath)));
});

extractorStream.pipe(untarStream).on('finish', () => {
const end = () => {
const error = this.validateError;
const hexDigest = this.validateIntegrity ? this.validateIntegrity.hexDigest() : '';
if (
Expand Down Expand Up @@ -189,7 +189,12 @@ export default class TarballFetcher extends BaseFetcher {
return resolve({
hash: this.hash || hexDigest,
});
};

extractorStream.pipe(untarStream).on('finish', () => {
setTimeout(end, 1000);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the delay I was mentionning. Removing it seems to cause issues with:

D:\yarn\bin\yarn.cmd jest -t "it should install the dependencies of any dependency fetched from the filesystem"

It seems like the package.json for this package isn't flushed until after the finish event is emitted, which causes issues when we try to read the dependencies later on. Yarn fallback to "no dependency", which makes the test crash because it test that the dependency can be loaded.

});

return {validateStream, extractorStream};
}

Expand Down
4 changes: 2 additions & 2 deletions src/util/generate-pnp-map-api.tpl.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ const isDirRegExp = /\/$/;
const isStrictRegExp = /^\.{0,2}\//;

// Splits a require request into its components, or return null if the request is a file path
const pathRegExp = /^(?!\.{0,2}(?:\/|$))((?:@[^\/]+\/)?[^\/]+)\/?(.*|)$/;
const pathRegExp = /^(?![a-zA-Z]:[\\\/]|\\\\|\.{0,2}(?:\/|$))((?:@[^\/]+\/)?[^\/]+)\/?(.*|)$/;

// Keep a reference around ("module" is a common name in this context, so better rename it to something more significant)
const pnpModule = module;
Expand Down Expand Up @@ -323,7 +323,7 @@ exports.resolveToUnqualified = function resolveToUnqualified(request, issuer, {c
// contain multiple levels of dependencies (ie. a yarn.lock inside a subfolder of a yarn.lock). This is
// typically solved using workspaces, but not all of them have been converted already.

if (ignorePattern && ignorePattern.test(issuer)) {
if (ignorePattern && ignorePattern.test(normalizePath(path.normalize(issuer)))) {
const result = callNativeResolution(request, issuer);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on how the pathRegExp is used you might be able to gate on path.parse()


if (result === false) {
Expand Down