Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature]: Debugging Reporter #2910

Merged
merged 27 commits into from
Jan 29, 2021
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
ba9e55a
Add script command. Remove sample backend script
sirugh Dec 16, 2020
c0a6e56
Initial
sirugh Dec 16, 2020
abd3484
More
sirugh Dec 17, 2020
eb1a6cf
cleanup
sirugh Dec 17, 2020
3663c4b
remove unused const
sirugh Dec 17, 2020
ed501b7
Update bug report template
sirugh Dec 17, 2020
0a4bfa4
Update bug report template
sirugh Dec 17, 2020
4072227
Remove magento version check since not all backends will expose it
sirugh Dec 17, 2020
750d7b4
process.version() -> process.version
revanth0212 Dec 21, 2020
7c73482
Parsing package-lock.json directly to get version details.
revanth0212 Dec 21, 2020
cc77b05
Using @yarnpkg/lockfile to generate dep-version list.
revanth0212 Dec 22, 2020
e9a314d
Added try catch around os version check.
revanth0212 Dec 22, 2020
c858cf5
Merge branch 'develop' into rugh/hackathon-2020-build-report
revanth0212 Dec 22, 2020
65d300e
Removed unnecessary code.
revanth0212 Dec 22, 2020
a44a00c
Merge branch 'rugh/hackathon-2020-build-report' of https://github.com…
revanth0212 Dec 22, 2020
045811b
Merge branch 'develop' into rugh/hackathon-2020-build-report
sirugh Jan 7, 2021
6a97577
Version updates.
revanth0212 Jan 7, 2021
f8559fb
Added supprort for multiple versions.
revanth0212 Jan 7, 2021
789c124
Minor logger change.
revanth0212 Jan 7, 2021
7c7ae86
Added mock fixtures.
revanth0212 Jan 8, 2021
abec157
Added initial test work.
revanth0212 Jan 8, 2021
ae7a30a
Added yarn.lock tests.
revanth0212 Jan 11, 2021
535cfd6
Added npm related tests.
revanth0212 Jan 11, 2021
4ddb8f1
Added other tests.
revanth0212 Jan 11, 2021
4ea451c
Added support for @adobe packages scanning.
revanth0212 Jan 11, 2021
a061195
Run tests if env is valid.
revanth0212 Jan 12, 2021
d527baa
Merge branch 'develop' into rugh/hackathon-2020-build-report
dpatil-magento Jan 28, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 28 additions & 18 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: Bug Report
about: Report a bug you've found in our code!
title: "[bug]: A short, simple sentence describing the bug"
title: '[bug]: A short, simple sentence describing the bug'
labels: bug
assignees: ''
---
Expand All @@ -19,44 +19,54 @@ Feel free to remove this section before creating this issue.
-->

**Describe the bug**

A clear and concise description of what the bug is.

**To reproduce**

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

**Expected behavior**

A clear and concise description of what you expected to happen.

**Screenshots**

If applicable, add screenshots to help explain your problem.

**Additional context**
Add any other context about the problem here.

Paste the output of `yarn build:report`, and any other context about the problem, here.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️


**Possible solutions**

Add any ideas about possible solutions to the problem here.

**Please complete the following device information:**
- Device [e.g. iPhone6, PC, Mac, Pixel3]:
- OS [e.g. iOS8.1, Windows 10]:
- Browser [e.g. Chrome, Safari]:
- Browser Version [e.g. 22]:
- Magento Version:
- PWA Studio Version:
- NPM version `npm -v`:
- Node Version `node -v`:

- Device [e.g. iPhone6, PC, Mac, Pixel3]:
- OS [e.g. iOS8.1, Windows 10]:
- Browser [e.g. Chrome, Safari]:
- Browser Version [e.g. 22]:
- Magento Version:
- PWA Studio Version:
- NPM version `npm -v`:
- Node Version `node -v`:

<!-- Complete the following sections to help us apply appropriate labels! -->

**Please let us know what packages this bug is in regards to:**
- [ ] `venia-concept`
- [ ] `venia-ui`
- [ ] `pwa-buildpack`
- [ ] `peregrine`
- [ ] `pwa-devdocs`
- [ ] `upward-js`
- [ ] `upward-spec`
- [ ] `create-pwa`

- [ ] `venia-concept`
- [ ] `venia-ui`
- [ ] `pwa-buildpack`
- [ ] `peregrine`
- [ ] `pwa-devdocs`
- [ ] `upward-js`
- [ ] `upward-spec`
- [ ] `create-pwa`
212 changes: 212 additions & 0 deletions packages/pwa-buildpack/lib/cli/generate-build-report.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
const { existsSync } = require('fs');
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');
const https = require('https');
const fetch = require('node-fetch');
const agent = new https.Agent({
rejectUnauthorized: false
});

const fetchWithAgent = async url => {
return await fetch(url, { agent });
};

const prettyLogger = require('../util/pretty-logger');
const { loadEnvironment } = require('../Utilities');

const { uniqBy } = require('lodash');
const { sampleBackends: defaultSampleBackends } = require('./create-project');

const removeDuplicateBackends = backendEnvironments =>
uniqBy(backendEnvironments, 'url');

const fetchSampleBackendUrls = async () => {
try {
const res = await fetch(
'https://fvp0esmt8f.execute-api.us-east-1.amazonaws.com/default/getSampleBackends'
);
const { sampleBackends } = await res.json();

return removeDuplicateBackends([
...sampleBackends.environments,
...defaultSampleBackends.environments
]).map(({ url }) => url);
} catch {
return defaultSampleBackends.environments.map(({ url }) => url);
}
};

const cwd = process.cwd();

function inspectDependencies(package) {
prettyLogger.info('Inspecting Dependencies');

// Inspect all '@magento/...' dependencies.
const magentoDependencies = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirugh while writing tests for this file, it hit me, we should be capturing all adobe and magento packages right, not just magento. What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depends on what you really want to glean from this information. I think we have at least one adobe dependency so might as well throw that in :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We actually use two:

@adobe/adobe-client-data-layer
@adobe/apollo-link-mutation-queue

...Object.keys(package.dependencies),
...Object.keys(package.devDependencies)
].filter(dependency => dependency.startsWith('@magento/'));

let dependencies;
if (existsSync(path.resolve(cwd, 'yarn.lock'))) {
// using yarn
dependencies = getYarnDependencies(magentoDependencies);
prettyLogger.log(
`Found ${dependencies.size} @magento dependencies in yarn.lock`
);
} else if (existsSync(path.resolve(cwd, 'package-lock.json'))) {
// using npm
dependencies = getNpmDependencies(magentoDependencies);
prettyLogger.log(
`Found ${
dependencies.size
} @magento dependencies in package-lock.json`
);
} else {
throw new Error(
'Dependencies have not been installed, please run `yarn install` or npm install` then generate the build report again.'
);
}

function logMapElements(value, key) {
prettyLogger.log(`${key} @ ${value}`);
}

dependencies.forEach(logMapElements);
}

function getYarnDependencies(dependencies) {
const dependencyMap = new Map();

dependencies.map(packageName => {
const whyBuffer = spawnSync('yarn', ['why', packageName]);
const grepBuffer = spawnSync('grep', ['Found'], {
input: whyBuffer.stdout
});

// [ 'info \r=> Found "@magento/pwa-buildpack@7.0.0"', '' ]
const outputArray = grepBuffer.stdout.toString().split('\n');

// [ '7.0.0' ]
const parsedOutputArray = outputArray
.filter(output => output.length > 0)
.map(output => output.split('@')[2].replace('"', ''));

dependencyMap.set(packageName, parsedOutputArray.toString());
});

return dependencyMap;
}

function getNpmDependencies(dependencies) {
const dependencyMap = new Map();

dependencies.map(packageName => {
const listbuffer = spawnSync('npm', ['list', packageName]);
const grepBuffer = spawnSync('grep', ['└──'], {
input: listbuffer.stdout
});

// TODO: handle multiple versions installed when not dupe. ie "npm ls core-js"
const [, , version] = grepBuffer.stdout
.toString()
.split('\n') // ["│ └── @magento/upward-js@5.0.0 deduped", "└── @magento/upward-js@5.0.0"]
.filter(text => !text.includes('deduped'))[0] // ["└── @magento/upward-js@5.0.0"]
.split('@'); // [ '└── ', 'magento/upward-js', '5.0.0' ]

dependencyMap.set(packageName, version);
});

return dependencyMap;
}

async function inspectBackend() {
prettyLogger.info('Inspecting Magento Backend');
const [projectConfig, sampleBackends] = await Promise.all([
loadEnvironment(
// Load .env from root
path.resolve(cwd)
),
fetchSampleBackendUrls()
]);

if (projectConfig.error) {
throw projectConfig.error;
}

const {
env: { MAGENTO_BACKEND_URL }
} = projectConfig;

if (sampleBackends.includes(MAGENTO_BACKEND_URL)) {
prettyLogger.log('Using sample backend: ', MAGENTO_BACKEND_URL);
} else {
prettyLogger.log('Not using sample backend.');
}

// Upcheck
try {
await fetchWithAgent(MAGENTO_BACKEND_URL);
prettyLogger.log('Backend is UP!');
} catch (e) {
prettyLogger.log('Backend is DOWN!');
prettyLogger.error('Reason:', e);
}

// Backend Version
try {
const res = await fetchWithAgent(
`${MAGENTO_BACKEND_URL}/magento_version`
);
prettyLogger.log('Magento Version:', await res.text());
} catch (e) {
prettyLogger.warn(
'Unable to determine Magento version - ensure that the Magento_Version route is enabled when generating this report.'
);
prettyLogger.error('Reason:', e);
}
}

function inspectBuildEnv() {
const osVersion = os.version();
Copy link
Contributor

@revanth0212 revanth0212 Dec 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to be careful of this. os.version is added in node 13.

Failed in my local machine since I was running node 10. Do we recommend a minimum version of node for using PWA Studio?

image

Started working after I moved to the latest stable (14.15.3).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, you're right! Our monorepo package.json requires at least 10, but if this doesn't work in 10 we need a fall back (upgrading to minimum 12 is definitely out of scope).

const nodeVersion = process.version();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure of this API? I checked the node docs and realized process.version is a string, not a function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another good catch. I wonder how I got this to work. Maybe I had it written when I ran it for the screenshot and forgot to push? 🤷 Thanks for fixing :)

const versionBuffer = spawnSync('npm', ['-v']);
const npmVersion = versionBuffer.stdout.toString();

prettyLogger.info('Inspecting System');
prettyLogger.log('OS:', osVersion);
prettyLogger.log('Node Version:', nodeVersion);
prettyLogger.log('NPM Version:', npmVersion);
}
/**
* main
*/
async function buildpackCli() {
const package = require(path.resolve(cwd, 'package.json'));
prettyLogger.info(
`Generating build report for ${package.name}@${
package.version
}. This may take a moment.`
);
prettyLogger.log('\n');

inspectDependencies(package);

prettyLogger.log('\n');

await inspectBackend();

prettyLogger.log('\n');

inspectBuildEnv();

prettyLogger.log('\n');
}

module.exports.command = 'generate-build-report';

module.exports.describe =
'Generates a report of general build information that can be used when debugging an issue.';

module.exports.handler = buildpackCli;
1 change: 1 addition & 0 deletions packages/venia-concept/_buildpack/create.js
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ async function createProjectFromVenia({ fs, tasks, options }) {
'build:analyze',
'build:dev',
'build:prod',
'build:report',
'clean',
'download-schema',
'lint',
Expand Down
1 change: 1 addition & 0 deletions packages/venia-concept/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
"build:debug": "node --inspect-brk ./node_modules/.bin/webpack --no-progress --env.mode",
"build:dev": "yarn run clean && yarn run validate-queries && webpack --no-progress --env.mode development",
"build:prod": "yarn run clean && webpack --no-progress --env.mode production",
"build:report": "buildpack generate-build-report",
"buildpack": "buildpack",
"clean": "rimraf dist",
"download-schema": "graphql get-schema --project venia --insecure",
Expand Down
2 changes: 1 addition & 1 deletion packages/venia-ui/upward.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ veniaResponse:
# handled by the top-level 'veniaProxy' object, which is a ProxyResolver
# that passes the request through to the backing Magento server.
- matches: request.url.pathname
pattern: '^/(graphql|rest|media)(/|$)'
pattern: '^/(magento_version|graphql|rest|media)(/|$)'
use: veniaProxy
- matches: request.url.pathname
pattern: '^/(robots\.txt|favicon\.ico|manifest\.json)'
Expand Down