Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V3.2: Adapt to the new Datacamp changes #47

Merged
merged 4 commits into from
Dec 4, 2021
Merged

V3.2: Adapt to the new Datacamp changes #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a1e0f74
Fix fetching completed courses/tracks
TRoboto Dec 3, 2021
f13dcd3
Fix issue when login with username and password
TRoboto Dec 3, 2021
6068469
Update readme
TRoboto Dec 3, 2021
ee00a92
Change downloads badge
TRoboto Dec 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 3 additions & 8 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![PyPI version](https://badge.fury.io/py/datacamp-downloader.svg)](https://pypi.org/project/datacamp-downloader/)
[![Documentation Status](https://readthedocs.org/projects/ansicolortags/badge/?version=latest)](https://github.com/TRoboto/datacamp-downloader/blob/master/docs.md)

[![Downloads](https://pepy.tech/badge/datacamp-downloader/week)](https://pepy.tech/project/datacamp-downloader)
[![Downloads](https://pepy.tech/badge/datacamp-downloader)](https://pepy.tech/project/datacamp-downloader)
[![GitHub stars](https://img.shields.io/github/stars/TRoboto/datacamp-downloader)](https://github.com/TRoboto/datacamp-downloader/stargazers)
[![GitHub forks](https://img.shields.io/github/forks/TRoboto/datacamp-downloader)](https://github.com/TRoboto/datacamp-downloader/network/members)
[![GitHub contributors](https://img.shields.io/github/contributors/TRoboto/datacamp-downloader)](https://github.com/TRoboto/datacamp-downloader/graphs/contributors)
Expand All @@ -27,7 +27,8 @@

## Update

Datacamp Downloader V3.1 is now available. The major change is that the tool now uses selenium for the backend. See changelog for version [3.0](https://github.com/TRoboto/datacamp-downloader/pull/39) and [3.1](https://github.com/TRoboto/datacamp-downloader/pull/42).
Datacamp Downloader V3.2 is now available. The major change is that the tool now uses selenium for the backend. See changelog for version [3.0](https://github.com/TRoboto/datacamp-downloader/pull/39), [3.1](https://github.com/TRoboto/datacamp-downloader/pull/42)
and [3.2](https://github.com/TRoboto/datacamp-downloader/pull/47).

## Description

Expand All @@ -53,12 +54,6 @@ If you use pip, you can install datacamp-downloader with:
pip install datacamp-downloader
```

If you want to install the previous version that doesn't use `selenium`, run:

```
pip install datacamp-downloader==2.2
```

### From source

You can directly clone this repo and install the tool with:
Expand Down
1 change: 1 addition & 0 deletions src/datacamp_downloader/constants.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
SESSION_FILE = tempfile.gettempdir() + "/.datacamp.v3"

PROFILE_URL = "https://www.datacamp.com/profile/{slug}"
PROFILE_DATA_URL = "https://www.datacamp.com/api/public/users/{slug}"
COURSE_DETAILS_API = "https://campus-api.datacamp.com/api/courses/{id}/"
EXERCISE_DETAILS_API = "https://campus-api.datacamp.com/api/exercise/{id}"
VIDEO_DETAILS_API = "https://projector.datacamp.com/api/videos/{hash}"
Expand Down
18 changes: 9 additions & 9 deletions src/datacamp_downloader/datacamp_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
LANGMAP,
LOGIN_DETAILS_URL,
LOGIN_URL,
PROFILE_URL,
PROFILE_DATA_URL,
PROGRESS_API,
VIDEO_DETAILS_API,
)
Expand Down Expand Up @@ -122,7 +122,9 @@ def login(self, username, password):

submit_button = self.session.get_element_by_xpath('//input[@tabindex="4"]')
submit_button.click()
self.session.wait_for_element_by_css_selector("#__next", "#flash_messages")
self.session.wait_for_element_by_css_selector(
"#mfe-composer-layout", "#flash_messages"
)

page = self.session.driver.page_source
if not page or "/users/sign_up" in page:
Expand All @@ -148,12 +150,10 @@ def set_token(self, token):

def get_profile_data(self):
if not self.profile_data:
profile = self.session.get(PROFILE_URL.format(slug=self.login_data["slug"]))
self.session.driver.minimize_window()
soup = BeautifulSoup(profile, "html.parser")
self.profile_data = self.session.to_json(
soup.find(id="__NEXT_DATA__").string
self.profile_data = self.session.get_json(
PROFILE_DATA_URL.format(slug=self.login_data["slug"])
)
self.session.driver.minimize_window()
return self.profile_data

@login_required
Expand Down Expand Up @@ -379,7 +379,7 @@ def get_completed_tracks(self, refresh=False):
self.tracks = []

data = self.get_profile_data()
completed_tracks = data["props"]["pageProps"]["completed_tracks"]
completed_tracks = data["completed_tracks"]
for i, track in enumerate(completed_tracks, 1):
self.tracks.append(Track(f"t{i}", track["title"].strip(), track["url"]))
all_courses = set()
Expand Down Expand Up @@ -407,7 +407,7 @@ def get_completed_courses(self, refresh=False):
self.courses = []

data = self.get_profile_data()
completed_courses = data["props"]["pageProps"]["completed_courses"]
completed_courses = data["completed_courses"]
for course in completed_courses:
fetched_course = self.get_course(course["id"])
if not fetched_course:
Expand Down