Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]: Debugging Reporter #2910

Merged
merged 27 commits into from
Jan 29, 2021
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ba9e55a
Add script command. Remove sample backend script
sirugh Dec 16, 2020
c0a6e56
Initial
sirugh Dec 16, 2020
abd3484
More
sirugh Dec 17, 2020
eb1a6cf
cleanup
sirugh Dec 17, 2020
3663c4b
remove unused const
sirugh Dec 17, 2020
ed501b7
Update bug report template
sirugh Dec 17, 2020
0a4bfa4
Update bug report template
sirugh Dec 17, 2020
4072227
Remove magento version check since not all backends will expose it
sirugh Dec 17, 2020
750d7b4
process.version() -> process.version
revanth0212 Dec 21, 2020
7c73482
Parsing package-lock.json directly to get version details.
revanth0212 Dec 21, 2020
cc77b05
Using @yarnpkg/lockfile to generate dep-version list.
revanth0212 Dec 22, 2020
e9a314d
Added try catch around os version check.
revanth0212 Dec 22, 2020
c858cf5
Merge branch 'develop' into rugh/hackathon-2020-build-report
revanth0212 Dec 22, 2020
65d300e
Removed unnecessary code.
revanth0212 Dec 22, 2020
a44a00c
Merge branch 'rugh/hackathon-2020-build-report' of https://github.com…
revanth0212 Dec 22, 2020
045811b
Merge branch 'develop' into rugh/hackathon-2020-build-report
sirugh Jan 7, 2021
6a97577
Version updates.
revanth0212 Jan 7, 2021
f8559fb
Added supprort for multiple versions.
revanth0212 Jan 7, 2021
789c124
Minor logger change.
revanth0212 Jan 7, 2021
7c7ae86
Added mock fixtures.
revanth0212 Jan 8, 2021
abec157
Added initial test work.
revanth0212 Jan 8, 2021
ae7a30a
Added yarn.lock tests.
revanth0212 Jan 11, 2021
535cfd6
Added npm related tests.
revanth0212 Jan 11, 2021
4ddb8f1
Added other tests.
revanth0212 Jan 11, 2021
4ea451c
Added support for @adobe packages scanning.
revanth0212 Jan 11, 2021
a061195
Run tests if env is valid.
revanth0212 Jan 12, 2021
d527baa
Merge branch 'develop' into rugh/hackathon-2020-build-report
dpatil-magento Jan 28, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
210 changes: 210 additions & 0 deletions packages/pwa-buildpack/lib/cli/generate-build-report.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
const { existsSync } = require('fs');
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');
const https = require('https');
const fetch = require('node-fetch');
const agent = new https.Agent({
rejectUnauthorized: false
});

const fetchWithAgent = async url => {
return await fetch(url, { agent });
};

const prettyLogger = require('../util/pretty-logger');
const { loadEnvironment } = require('../Utilities');

const { uniqBy } = require('lodash');
const { sampleBackends: defaultSampleBackends } = require('./create-project');

const removeDuplicateBackends = backendEnvironments =>
uniqBy(backendEnvironments, 'url');

const fetchSampleBackendUrls = async () => {
try {
const res = await fetch(
'https://fvp0esmt8f.execute-api.us-east-1.amazonaws.com/default/getSampleBackends'
);
const { sampleBackends } = await res.json();

return removeDuplicateBackends([
...sampleBackends.environments,
...defaultSampleBackends.environments
]).map(({ url }) => url);
} catch {
return defaultSampleBackends.environments.map(({ url }) => url);
}
};

const cwd = process.cwd();

function inspectDependencies(package) {
prettyLogger.info('Inspecting Dependencies');

// Inspect all '@magento/...' dependencies.
const magentoDependencies = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirugh while writing tests for this file, it hit me, we should be capturing all adobe and magento packages right, not just magento. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends on what you really want to glean from this information. I think we have at least one adobe dependency so might as well throw that in :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually use two:

@adobe/adobe-client-data-layer
@adobe/apollo-link-mutation-queue

...Object.keys(package.dependencies),
...Object.keys(package.devDependencies)
].filter(dependency => dependency.startsWith('@magento/'));

let dependencies;
if (existsSync(path.resolve(cwd, 'yarn.lock'))) {
// using yarn
dependencies = getYarnDependencies(magentoDependencies);
prettyLogger.log(
`Found ${dependencies.size} @magento dependencies in yarn.lock`
);
} else if (existsSync(path.resolve(cwd, 'package-lock.json'))) {
// using npm
dependencies = getNpmDependencies(magentoDependencies);
prettyLogger.log(
`Found ${
dependencies.size
} @magento dependencies in package-lock.json`
);
} else {
throw new Error(
'Dependencies have not been installed, please run `yarn install` or npm install` then generate the build report again.'
);
}

function logMapElements(value, key) {
prettyLogger.log(`${key} @ ${value}`);
}

dependencies.forEach(logMapElements);
}

function getYarnDependencies(dependencies) {
const dependencyMap = new Map();

dependencies.map(packageName => {
const whyBuffer = spawnSync('yarn', ['why', packageName]);
const grepBuffer = spawnSync('grep', ['Found'], {
input: whyBuffer.stdout
});

// [ 'info \r=> Found "@magento/pwa-buildpack@7.0.0"', '' ]
const outputArray = grepBuffer.stdout.toString().split('\n');

// [ '7.0.0' ]
const parsedOutputArray = outputArray
.filter(output => output.length > 0)
.map(output => output.split('@')[2].replace('"', ''));

dependencyMap.set(packageName, parsedOutputArray.toString());
});

return dependencyMap;
}

function getNpmDependencies(dependencies) {
const dependencyMap = new Map();

dependencies.map(packageName => {
const listbuffer = spawnSync('npm', ['list', packageName]);
const grepBuffer = spawnSync('grep', ['└──'], {
input: listbuffer.stdout
});

// TODO: handle multiple versions installed when not dupe. ie "npm ls core-js"
const [, , version] = grepBuffer.stdout
.toString()
.split('\n') // ["│ └── @magento/upward-js@5.0.0 deduped", "└── @magento/upward-js@5.0.0"]
.filter(text => !text.includes('deduped'))[0] // ["└── @magento/upward-js@5.0.0"]
.split('@'); // [ '└── ', 'magento/upward-js', '5.0.0' ]

dependencyMap.set(packageName, version);
});

return dependencyMap;
}

async function inspectBackend() {
prettyLogger.info('Inspecting Magento Backend');
const [projectConfig, sampleBackends] = await Promise.all([
loadEnvironment(
// Load .env from root
path.resolve(cwd)
),
fetchSampleBackendUrls()
]);

if (projectConfig.error) {
throw projectConfig.error;
}

const {
env: { MAGENTO_BACKEND_URL }
} = projectConfig;

if (sampleBackends.includes(MAGENTO_BACKEND_URL)) {
prettyLogger.log('Using sample backend: ', MAGENTO_BACKEND_URL);
} else {
prettyLogger.log('Not using sample backend.');
}

// Upcheck
try {
const res = await fetchWithAgent(MAGENTO_BACKEND_URL);
prettyLogger.log('Backend is UP!');
} catch (e) {
prettyLogger.log('Backend is DOWN!');
prettyLogger.error('Reason:', e);
}

// Backend Version
try {
const res = await fetchWithAgent(
`${MAGENTO_BACKEND_URL}/magento_version`
);
prettyLogger.log('Magento Version:', await res.text());
} catch (e) {
prettyLogger.warn(
'Unable to determine Magento version - ensure that the Magento_Version route is enabled when generating this report.'
);
prettyLogger.error('Reason:', e);
}
// https://venia-cicd-lrov2hi-mfwmkrjfqvbjk.us-4.magentosite.cloud/magento_version
}

function inspectBuildEnv() {
prettyLogger.info('Inspecting System');
prettyLogger.log('OS:', os.version());

prettyLogger.log('Node Version:', process.version);
const versionBuffer = spawnSync('npm', ['-v']);
prettyLogger.log('NPM Version:', versionBuffer.stdout.toString());
}
/**
* main
*/
async function buildpackCli() {
const package = require(path.resolve(cwd, 'package.json'));
prettyLogger.info(
`Generating build report for ${package.name}@${
package.version
}. This may take a moment.`
);
prettyLogger.log('\n');

inspectDependencies(package);

prettyLogger.log('\n');

await inspectBackend();

prettyLogger.log('\n');

inspectBuildEnv();

prettyLogger.log('\n');
}

module.exports.command = 'generate-build-report';

module.exports.describe =
'Generates a report of general build information that can be used when debugging an issue.';

module.exports.handler = buildpackCli;
1 change: 1 addition & 0 deletions packages/venia-concept/_buildpack/create.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ async function createProjectFromVenia({ fs, tasks, options }) {
'build:analyze',
'build:dev',
'build:prod',
'build:report',
'clean',
'download-schema',
'lint',
Expand Down
1 change: 1 addition & 0 deletions packages/venia-concept/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"build:debug": "node --inspect-brk ./node_modules/.bin/webpack --no-progress --env.mode",
"build:dev": "yarn run clean && yarn run validate-queries && webpack --no-progress --env.mode development",
"build:prod": "yarn run clean && webpack --no-progress --env.mode production",
"build:report": "buildpack generate-build-report",
"buildpack": "buildpack",
"clean": "rimraf dist",
"download-schema": "graphql get-schema --project venia --insecure",
Expand Down
2 changes: 1 addition & 1 deletion packages/venia-ui/upward.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ veniaResponse:
# handled by the top-level 'veniaProxy' object, which is a ProxyResolver
# that passes the request through to the backing Magento server.
- matches: request.url.pathname
pattern: '^/(graphql|rest|media)(/|$)'
pattern: '^/(magento_version|graphql|rest|media)(/|$)'
use: veniaProxy
- matches: request.url.pathname
pattern: '^/(robots\.txt|favicon\.ico|manifest\.json)'
Expand Down