Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content type of the sitemap.xml is missing #467

Closed
lipis opened this issue Feb 21, 2018 · 22 comments
Closed

Content type of the sitemap.xml is missing #467

lipis opened this issue Feb 21, 2018 · 22 comments
Labels
better engineering Not a bug or feature request good first issue If you are just getting started with Docusaurus, this issue should be a good place to begin.

Comments

@lipis
Copy link

lipis commented Feb 21, 2018

Should be: application/xml

https://stackoverflow.com/a/3272572/8418

@JoelMarcey JoelMarcey added the good first issue If you are just getting started with Docusaurus, this issue should be a good place to begin. label Feb 26, 2018
@Glavin001
Copy link
Contributor

@s-pace from Algolia also reported sitemap.xml is not compliant with https://www.sitemaps.org/protocol.html

See algolia/docsearch-configs#312 (comment)

@rizafahmi
Copy link

Let me try to fix this

@yangshun
Copy link
Contributor

@rizafahmi It's yours!

I've added an "issue-claimed" label so that others will know not to start work on the issue. If you change your mind about the issue, no worries! Just let me know so that I can remove the label and free it up for someone else to claim.

I'll check in with you periodically so that we can keep the task updated with the progress.

@rizafahmi
Copy link

Thanks @yangshun !

@rizafahmi
Copy link

Hi @lipis can you please confirm that the content-type is not application/xml?

I tried it with curl & httpie, it's already application/xml.

http http://localhost:3000/sitemap.xml -h
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 3060
Content-Type: application/xml; charset=utf-8
Date: Sun, 15 Apr 2018 14:48:38 GMT
ETag: W/"bf4-xyO2OFprLzDPuBKeUuyaw33D+Fk"
X-Powered-By: Express

@lipis
Copy link
Author

lipis commented Apr 15, 2018

screen shot 2018-04-15 at 19 04 18

Type: Document

And it renders it wrong in the browser..

Compare the two in the browser:

@JoelMarcey
Copy link
Contributor

So we use a third party npm library for the sitemap. https://github.com/ekalinin/sitemap.js (or https://www.npmjs.com/package/sitemap).

https://github.com/facebook/Docusaurus/blob/master/lib/server/sitemap.js#L96 is where we set it. Maybe we are missing some options?

I do notice we are doing <url><loc>... if you view the source (view-source:https://docusaurus.io/sitemap.xml)

instead of

<sitemap><loc>...

as in https://www.google.com/sitemap.xml

@s-pace
Copy link

s-pace commented Apr 16, 2018

👋 @JoelMarcey ,

Thanks for the details.

One of the main point is to be compliant with the official standard. Definitely possible with the npm package.

Happy to help if needed.

Cheers

@rizafahmi
Copy link

@lipis both docusaurus and google was document on my side.
screen shot 2018-04-16 at 6 12 27 pm
screen shot 2018-04-16 at 6 12 33 pm

But you're right about wrong rendering part. I'll check it out.

@lipis
Copy link
Author

lipis commented Apr 16, 2018

@rizafahmi My bad.. check the Response Headers. The content-type is missing.

screen shot 2018-04-16 at 13 21 58

@rizafahmi
Copy link

ah. i see. Thanks for the clarification @lipis

@rizafahmi
Copy link

Sorry @lipis to bother you again. There is actually content-type on the response headers.

screen shot 2018-04-17 at 10 50 20 am

I did validate the sitemap over https://validator.w3.org and https://www.xml-sitemaps.com/validate-xml-sitemap.html and all says valid.
Tried to look up the code as well, seems nothing wrong with the code. To be precise, I don't know if there something wrong with the code.
I also tried to use sitemap package with plain express app and it's produce the same thing.

@JoelMarcey
Copy link
Contributor

@s-pace Thanks for chiming in here. Hope all is well. Given what @rizafahmi says in the comment just above, do you think we what we have is already good and we do not need to make any changes?

@lipis
Copy link
Author

lipis commented Apr 17, 2018

Not sure.. when I'm checking with cURL it's there.. but not in my Chrome :(

So something is "wrong" and the best indicator is that it's not rendered as XML but as text.. not the most important thing in the world.. but still :)

@s-pace
Copy link

s-pace commented Apr 17, 2018

👋 @JoelMarcey My pleasure, hope you are doing well too.

@rizafahmi Happy to try it with our tool. Could you provide me a testing link? 🙏

@JoelMarcey
Copy link
Contributor

@s-pace This is the link that is being debated whether is wrong or correct? https://docusaurus.io/sitemap.xml

This is a link that is assumed correct - https://www.google.com/sitemap.xml

s-pace pushed a commit to algolia/docsearch-configs that referenced this issue Apr 17, 2018
@s-pace
Copy link

s-pace commented Apr 17, 2018

@JoelMarcey

Works 💯. I have made an example:

  • Copy the following CSS/JS snippets and add them to your page
<!-- at the end of the HEAD -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.css" />

<!-- at the end of the BODY -->
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/docsearch.js@2/dist/cdn/docsearch.min.js"></script>
<script type="text/javascript"> docsearch({
  apiKey: '46899d51aa254ffb4fd068f3ae64ed40',
  indexName: 'docusaurus_demo',
  inputSelector: '### REPLACE ME ####',
  algoliaOptions: { 'facetFilters': ["lang:$LANG", "version:$VERSION", "tags:$TAGS"] },
  debug: false // Set debug to true if you want to inspect the dropdown
});
</script>
  • Add a search input in your page if you don't have any yet. Then update the inputSelector value in JS snippet to a CSS selector that targets your search input field.

  • Replace $LANG with the lang you want to search on.
    The list of possible lang is hardcoded in the config.
    So as of today you have: en

  • Replace $VERSION with the version you want to search on.
    The list of possible version is hardcoded in the config.
    So as of today you have: latest, next

  • Replace $TAGS with the tags you want to search on.
    The list of possible tags is hardcoded in the config.
    So as of today you have: blog

    For example if you want to refine the search to the lang "en" and the version "latest" and the tags "blog" just specify:

'facetFilters': ["lang:en", "version:latest", "tags:blog"]

We'd also be happy to get your feedback and thoughts about DocSearch - so we can continue to improve it.

Have a nice day :)

@JoelMarcey
Copy link
Contributor

@s-pace Thank you! 👍 I commented on the docsearch commit you made that I think you changed the wrong config file. Let me know if that was the case. If so, we can fix that 😄

In the case of Docusaurus, we probably only need to add the algoliaOptions field since we already have the search box, etc. Does that seem right?

@s-pace
Copy link

s-pace commented Apr 18, 2018

@JoelMarcey

Exactly! I use this index in order to show you how it would looks like. Let me know if I can merge it to the original one.

@JoelMarcey
Copy link
Contributor

Thanks @s-pace - I commented on the config changes over at algolia/docsearch-configs@db60b72 with a couple of questions. Assuming those are good, then I think we can merge it.

@endiliey endiliey added the better engineering Not a bug or feature request label Jun 2, 2018
@endiliey
Copy link
Contributor

endiliey commented Jun 4, 2018

Our sitemap is actually correct.
sitemap

The XML Viewer in chrome is wrong when we include alternate languages page on the sitemap.
Refer to ekalinin/sitemap.js#37

I tried https://www.w3schools.com/xml/xml_validator.asp & it's actually valid.

If we really want to get that beautiful XML Viewer, removing the alternate languages will do so
sdfgh

Edit: Closing this

@endiliey endiliey closed this as completed Jun 9, 2018
@storemey01
Copy link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
better engineering Not a bug or feature request good first issue If you are just getting started with Docusaurus, this issue should be a good place to begin.
Projects
None yet
Development

No branches or pull requests

8 participants