Stanford Question Answering Dataset

[zachgk]

Description

Add the dataset to the basic datasets.

References

• https://rajpurkar.github.io/SQuAD-explorer/

[ghost]

Hi @zachgk , can you please assign this to me? I am interested in taking this up.

[ghost]

Hi @zachgk , I am working on adding this dataset and understanding the general framework. I have one basic question about the dataset artifact's metadata.json. I see that the artifacts's uris are pinting to djl mlrepo ,so how are the artifacts added to this repo? Also, please also share some resources that I can refer to, if any. Thanks!

[frankfliu]

@ghost, once you have files ready, we can help you upload to s3 bucket. @lanking520 can provide more detail regarding format of metadata.json.
Maybe use our slack channel is a better way to work with you.

[ghost]

Thanks @frankfliu ! Will post any other clarifications on Slack! :-)

[lanking520]

@codelessricky I just started a quick implementation to bring your own dataset: You can make a start from this branch: https://github.com/awslabs/djl/tree/model-upload/model-uploader to try it out. Please just be aware this is WIP and feel free to raise PR if you find anywhere wrong and add more feature.

[WHALEEYE]

Hi @zachgk, I'm interested in this issue and I want to work on it, so I wonder if you can assign it to me? Thanks!

[zachgk]

You've got it @WHALEEYE. Feel free to post here or reach out to me if you have any questions

[WHALEEYE]

Hi @zachgk, I've encountered some problem while adding SQuAD to the project.

1. At first I want to inherit the TextDataset class, but then I found that the get() method inherited from RandomAccessDataset returns a Record object containing two NDArrays, while in SQuAD each record will contain a context, a question, a list of answers and a list of indexes (we referred to PyTorch), so this seems not fit the structure of Record, so now I get a little confused about which class should I inherit.
2. Should I preprocess the string in the data, like embedding, while preparing the data (like TensorFlow), or just give user a record with the context and questions being raw strings (like PyTorch)?

Thanks for your answer!

[zachgk]

We generally try to handle the preprocessing within the dataset class. There is a helper called TextData that we use to configure the dataset to know how to pre-process textual data, which should be pretty helpful to you. In fact, the TextDataset is just a wrapper around RandomAccessDataset that uses the TextData. You may want to take a look at the StanfordMovieReview dataset which is a good example of a text dataset. In summary, you probably want to inherit TextDataset but could also directly use RandomAccessDataset.

I think there may also be some confusion about Record. Record doesn't return an NDArray, but an NDList. We use the NDList as a tuple of arrays as well. So, your tuple might be something like Record(data=NDList(contextPreprocessed, questionPreprocessed), labels=NDList(answerPreprocessed, indices)).

[WHALEEYE]

I think I've got it. Thanks for answering!