Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create instagram_crawler.py #2509

Merged
merged 18 commits into from
Sep 29, 2020
Merged

Create instagram_crawler.py #2509

merged 18 commits into from
Sep 29, 2020

Conversation

yogeshwaran01
Copy link
Contributor

Describe your change:

Crawl the Instagram users information

  • Add an algorithm?
  • Fix a bug or typo in an existing algorithm?
  • Documentation change?

Checklist:

  • I have read CONTRIBUTING.md.
  • This pull request is all my own work -- I have not plagiarized.
  • I know that pull requests will not be merged if they fail the automated tests.
  • This PR only changes one algorithm file. To ease review, please open separate PRs for separate algorithms.
  • All new Python files are placed inside an existing directory.
  • All filenames are in all lowercase characters with no spaces or dashes.
  • All functions and variable names follow Python naming conventions.
  • All function parameters and return values are annotated with Python type hints.
  • All functions have doctests that pass the automated testing.
  • All new algorithms have a URL in its comments that points to Wikipedia or other similar explanation.
  • If this pull request resolves one or more open issues then the commit message contains Fixes: #{$ISSUE_NO}.

@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
3cd53cb 
Co-authored-by: Christian Clauss <cclauss@me.com>
@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
cd6f44e 
Co-authored-by: Christian Clauss <cclauss@me.com>
@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
52ec0f6 
Co-authored-by: Christian Clauss <cclauss@me.com>
@TravisBuddy
Copy link

Hey @yogeshwaran01,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: 30438200-024d-11eb-aba2-872ffb2742c8

@TravisBuddy
Copy link

Hey @yogeshwaran01,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: 304333e0-024d-11eb-aba2-872ffb2742c8

@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
66c157d 
Co-authored-by: Christian Clauss <cclauss@me.com>
@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
c50204c 
Co-authored-by: Christian Clauss <cclauss@me.com>
cclauss
cclauss requested changes Sep 29, 2020
View reviewed changes
@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
fcdf0eb 
Co-authored-by: Christian Clauss <cclauss@me.com>
@cclauss cclauss mentioned this pull request Sep 29, 2020
Closed
14 tasks
@yogeshwaran01 @cclauss
Update web_programming/instagram_crawler.py
6756207 
Co-authored-by: Christian Clauss <cclauss@me.com>
yogeshwaran01
yogeshwaran01 commented Sep 29, 2020
View reviewed changes
Copy link
Contributor Author

@yogeshwaran01 yogeshwaran01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@yogeshwaran01
Copy link
Contributor Author

Requested changes are commited.

yogeshwaran01
yogeshwaran01 commented Sep 29, 2020
View reviewed changes
web_programming/instagram_crawler.py Outdated
return info["username"]

@property
def profile_picture_url(self) -> str:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

requested changes done

@TravisBuddy
Copy link

Travis tests have failed

Hey @yogeshwaran01,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: e41c8a80-025a-11eb-aba2-872ffb2742c8

@TravisBuddy
Copy link

Hey @yogeshwaran01,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: 5f37dcd0-025e-11eb-aba2-872ffb2742c8

@yogeshwaran01 @dhruvmanila
Update web_programming/instagram_crawler.py
2d7c21a 
Co-authored-by: Dhruv <dhruvmanila@gmail.com>
@TravisBuddy
Copy link

Travis tests have failed

Hey @yogeshwaran01,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: 2a97cec0-0260-11eb-aba2-872ffb2742c8

@dhruvmanila
Copy link
Member

dhruvmanila commented Sep 29, 2020 •
edited
Loading

I think the user_agent differs with the platform and browser used.

From wikipedia:
Mozilla/[version] ([system and browser information]) [platform] ([platform details]) [extensions]

"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) "
     "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"

Here, you are using a user_agent assuming to be on Macintosh and the browsers to be either Chrome or Safari while our Travis tests are run on Linux. I don't have much idea regarding this topic so please do the research and update the user_agent accordingly.

Resources:
https://en.wikipedia.org/wiki/User_agent
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent

@cclauss
Copy link
Member

cclauss commented Sep 29, 2020

In this repo, we have https://pypi.org/project/fake-useragent/ installed.

@TravisBuddy
Copy link

Hey @yogeshwaran01,
Something went wrong with the build.

TravisCI finished with status errored, which means the build failed because of something unrelated to the tests, such as a problem with a dependency or the build process itself.

View build log

TravisBuddy Request Identifier: a86adc20-0265-11eb-aba2-872ffb2742c8

@dhruvmanila
Copy link
Member

Please do these small checks locally so that you don't waste your time editing over and over again.
Thank you for your cooperation 😁

  • black <filename>
  • isort <filename>
  • flake8 --ignore=E203,W503 --max-complexity=25 --max-line-length=88 --statistics --count <filename>
  • mypy --ignore-missing-imports <filename>
  • pytest --doctest-modules <filename>

@cclauss
Copy link
Member

cclauss commented Sep 29, 2020 •
edited
Loading

@dhruvmanila Are you interested in creating a pre-commit hook for us?

https://github.com/cclauss/itinerant-tester/blob/master/.pre-commit-config.yaml
https://github.com/cclauss/itinerant-tester/blob/master/.github/workflows/pre-commit.yml

@dhruvmanila
Copy link
Member

dhruvmanila commented Sep 29, 2020 •
edited
Loading

Sure, I'll take a look at it. I've never tried one, so it will be a fun challenge!

There are also those mypy and pytest errors that need to be fixed. I think this is a good idea for Hacktoberfest.

@TravisBuddy
Copy link

Travis tests have failed

Hey @yogeshwaran01,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: 8dc39980-0269-11eb-aba2-872ffb2742c8

@dhruvmanila
Copy link
Member

Hey, @yogeshwaran01, Please run this command locally and see what you're getting as the output:

pytest --doctest-modules instagram_crawler.py

@yogeshwaran01
Copy link
Contributor Author

pytest --doctest-modules instagram_crawler.py

It passed the test

3 passed in 9.11s

@yogeshwaran01
Copy link
Contributor Author

Wow, It worked! Thank you, @cclauss and @dhruvmanila.

@dhruvmanila
Copy link
Member

Your welcome! 🎉 😃
@cclauss Is it ok to have no tests?

cclauss
cclauss approved these changes Sep 29, 2020
View reviewed changes
Copy link
Member

@cclauss cclauss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. The doctest run locally for both @yogeshwaran01 and I but they fail on Travis CI.

@cclauss cclauss merged commit e6e2dc6 into TheAlgorithms:master Sep 29, 2020
@cclauss
Copy link
Member

cclauss commented Sep 29, 2020

There is a gold star for whoever can make the doctests pass under Travis CI.

@dhruvmanila dhruvmanila mentioned this pull request Sep 30, 2020
Merged
7 tasks
stokhos pushed a commit to stokhos/Python that referenced this pull request Jan 3, 2021
@yogeshwaran01 @cclauss
@dhruvmanila @stokhos
Create instagram_crawler.py (TheAlgorithms#2509)
3a49cbf 
* Create instagram_crawler.py

* codespell --ignore-words-list=followings

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update instagram_crawler.py

* Add doctests

* fixup! except (json.decoder.JSONDecodeError, KeyError):

* if getenv("CONTINUOUS_INTEGRATION"): return

* Update instagram_crawler.py

* Update web_programming/instagram_crawler.py

Co-authored-by: Dhruv <dhruvmanila@gmail.com>

* added fake_useragent

* Update instagram_crawler.py

* Comment out doctests

Co-authored-by: Christian Clauss <cclauss@me.com>
Co-authored-by: Dhruv <dhruvmanila@gmail.com>
Panquesito7 pushed a commit to Panquesito7/Python that referenced this pull request May 13, 2021
@yogeshwaran01 @cclauss @dhruvmanila
Create instagram_crawler.py (TheAlgorithms#2509)
b48fdab 
* Create instagram_crawler.py

* codespell --ignore-words-list=followings

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update web_programming/instagram_crawler.py

Co-authored-by: Christian Clauss <cclauss@me.com>

* Update instagram_crawler.py

* Add doctests

* fixup! except (json.decoder.JSONDecodeError, KeyError):

* if getenv("CONTINUOUS_INTEGRATION"): return

* Update instagram_crawler.py

* Update web_programming/instagram_crawler.py

Co-authored-by: Dhruv <dhruvmanila@gmail.com>

* added fake_useragent

* Update instagram_crawler.py

* Comment out doctests

Co-authored-by: Christian Clauss <cclauss@me.com>
Co-authored-by: Dhruv <dhruvmanila@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dhruvmanila dhruvmanila dhruvmanila left review comments

@cclauss cclauss cclauss approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yogeshwaran01 @TravisBuddy @dhruvmanila @cclauss