How to download an entire “bottomless” (AJAX) page?











up vote
0
down vote

favorite












Examples for bottomless pages:




  • YouTube comments (used to have pages, now bottomless. RePagination is powerless here.)

  • Quora.com answers.

  • Twitter feed.

  • Twitter tweet threads.

  • Twitter follow lists.

  • Instagram profiles.

  • Various other comment sections.

  • Reddit.com subreddits: List of threads.

  • (slightly different kind:) YouTube's “Show more replies” button that loads 10 comments at a time.

  • (deviation) Cascade all Quora subcomments.


One could scroll down (Windows: Press middle mouse key and move cursor down. A missing feature in Linux.) until reaching the bottom, but is there an automated way to do so?



How to download an entire bottomless page?



And if the page is literally bottomless such as Qz.com's website layout that jumps to the next article, maybe there should be a page limit of e.g. 20 pages.



What I attempt to achieve: Finding lost text on bottomless pages.






ajax web layout ajaxform







share|improve this question






















  • (personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
    – neverMind9
    2 days ago















up vote
0
down vote

favorite












Examples for bottomless pages:




  • YouTube comments (used to have pages, now bottomless. RePagination is powerless here.)

  • Quora.com answers.

  • Twitter feed.

  • Twitter tweet threads.

  • Twitter follow lists.

  • Instagram profiles.

  • Various other comment sections.

  • Reddit.com subreddits: List of threads.

  • (slightly different kind:) YouTube's “Show more replies” button that loads 10 comments at a time.

  • (deviation) Cascade all Quora subcomments.


One could scroll down (Windows: Press middle mouse key and move cursor down. A missing feature in Linux.) until reaching the bottom, but is there an automated way to do so?



How to download an entire bottomless page?



And if the page is literally bottomless such as Qz.com's website layout that jumps to the next article, maybe there should be a page limit of e.g. 20 pages.



What I attempt to achieve: Finding lost text on bottomless pages.






ajax web layout ajaxform







share|improve this question






















  • (personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
    – neverMind9
    2 days ago













up vote
0
down vote

favorite









up vote
0
down vote

favorite











Examples for bottomless pages:




  • YouTube comments (used to have pages, now bottomless. RePagination is powerless here.)

  • Quora.com answers.

  • Twitter feed.

  • Twitter tweet threads.

  • Twitter follow lists.

  • Instagram profiles.

  • Various other comment sections.

  • Reddit.com subreddits: List of threads.

  • (slightly different kind:) YouTube's “Show more replies” button that loads 10 comments at a time.

  • (deviation) Cascade all Quora subcomments.


One could scroll down (Windows: Press middle mouse key and move cursor down. A missing feature in Linux.) until reaching the bottom, but is there an automated way to do so?



How to download an entire bottomless page?



And if the page is literally bottomless such as Qz.com's website layout that jumps to the next article, maybe there should be a page limit of e.g. 20 pages.



What I attempt to achieve: Finding lost text on bottomless pages.






ajax web layout ajaxform







share|improve this question













Examples for bottomless pages:




  • YouTube comments (used to have pages, now bottomless. RePagination is powerless here.)

  • Quora.com answers.

  • Twitter feed.

  • Twitter tweet threads.

  • Twitter follow lists.

  • Instagram profiles.

  • Various other comment sections.

  • Reddit.com subreddits: List of threads.

  • (slightly different kind:) YouTube's “Show more replies” button that loads 10 comments at a time.

  • (deviation) Cascade all Quora subcomments.


One could scroll down (Windows: Press middle mouse key and move cursor down. A missing feature in Linux.) until reaching the bottom, but is there an automated way to do so?



How to download an entire bottomless page?



And if the page is literally bottomless such as Qz.com's website layout that jumps to the next article, maybe there should be a page limit of e.g. 20 pages.



What I attempt to achieve: Finding lost text on bottomless pages.






ajax web layout ajaxform




ajax web layout ajaxform






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 2 days ago









neverMind9

1097




1097












  • (personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
    – neverMind9
    2 days ago


















  • (personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
    – neverMind9
    2 days ago
















(personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
– neverMind9
2 days ago




(personal opinion not in question text but in comments) I must say that I could never quite stand YouTube's AJAX comments. Especially now, they added “show more comment” bulk and removed the /all_comments?v= page in late 2015.
– neverMind9
2 days ago

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373310%2fhow-to-download-an-entire-bottomless-ajax-page%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373310%2fhow-to-download-an-entire-bottomless-ajax-page%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

MongoDB - Not Authorized To Execute Command

Image
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I have successfully enabled authorization on MongoDB and I have created an account on the admin database and then I created an account for my database called test. The following connection string to connect to my test database works successfully: mongo --host 192.168.17.52 --port 27017 -u user1 -p password --authenticationDatabase test Only problem I have now is, I cannot execute commands such as: show dbs. I get the following error when I try to do so: "errmsg" : "not authorized on admin to execute command { listDatabases: 1.0, lsid: { id: UUID("a1d5bc0d-bc58-485e-b232-270758a89455") }, $db: "admin...
Read more

How to fix TextFormField cause rebuild widget in Flutter

Image
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box; } 0 I've got the same issue at #9280. I tried all solutions from that topic with/without GlobalKey for Scaffold and Form but the problem wasn't fixed. I tried to build a authentication form with login and signup. But the keyboard doesn't show when I tap the TextFormField at the first time. Then I try to input some letter from physical keyboard, the widget is rebuilt. Like this: login screen Nothing to be log when I run flutter run --verbose, so I logged manually. Every tapping on field, the widget called dispose -> init -> build. [√] Flutter (Channel beta, v1.0.0, on Microsoft Windows [Version 10.0.17763.195], ...
Read more

in spring boot 2.1 many test slices are not allowed anymore due to multiple @BootstrapWith

Image
1 I tried to upgrade a yummy sandwich made of two test slices (@JsonTest and @JdbcTest in my case, crunchy test code in between) adding spring boot 2.1 flavour to it. But it seems it was not much of a success. I cannot annotate my tests with many @...Test since they are now each bringing their own XxxTestContextBootstrapper. It used to work when they all used same SpringBootTestContextBootstrapper. @RunWith(SpringRunner.class) @JdbcTest @JsonTest public class Test { @Test public void test() { System.out.printn("Hello, World !"); } } The error I get from BootstrapUtils is illegalStateException : Configuration error: found multiple declarations of @BootstrapWith for test class I understand I might be doing something wrong here but is there an easy way I could load both Json and Jdbc contex...
Read more