Mixing
Running tests against code examples in a README.md?
Does anyone know of an open-source project or program for running tests against code examples in README.md?
A perennial problem, my documentation tends to drift out of date with the code. For example, a code snippet in the README.md will no longer work with the current version, and isn't caught until a new developer on-boards onto the project. Is it possible to include README.md code snippets in my test suite?
For example, the usage of say.nancat with sample params:
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
The program would initialize an environment with the '#' (not shown in README.md because the context is assumed), run the '>' line and pass/fail based on the next line. Simular to doctests in python.
Loads of people have the problem of keeping README.md and other docs current with the code, so I was hoping there was an off-the-shelf solution. I've looked (DuckDuckGo) to no avail.
node.js github markdown doctest
edited Oct 15 '18 at 2:07
jww
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
add a comment |
Does anyone know of an open-source project or program for running tests against code examples in README.md?
A perennial problem, my documentation tends to drift out of date with the code. For example, a code snippet in the README.md will no longer work with the current version, and isn't caught until a new developer on-boards onto the project. Is it possible to include README.md code snippets in my test suite?
For example, the usage of say.nancat with sample params:
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
The program would initialize an environment with the '#' (not shown in README.md because the context is assumed), run the '>' line and pass/fail based on the next line. Simular to doctests in python.
Loads of people have the problem of keeping README.md and other docs current with the code, so I was hoping there was an off-the-shelf solution. I've looked (DuckDuckGo) to no avail.
node.js github markdown doctest
edited Oct 15 '18 at 2:07
jww
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
add a comment |
Does anyone know of an open-source project or program for running tests against code examples in README.md?
A perennial problem, my documentation tends to drift out of date with the code. For example, a code snippet in the README.md will no longer work with the current version, and isn't caught until a new developer on-boards onto the project. Is it possible to include README.md code snippets in my test suite?
For example, the usage of say.nancat with sample params:
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
The program would initialize an environment with the '#' (not shown in README.md because the context is assumed), run the '>' line and pass/fail based on the next line. Simular to doctests in python.
Loads of people have the problem of keeping README.md and other docs current with the code, so I was hoping there was an off-the-shelf solution. I've looked (DuckDuckGo) to no avail.
node.js github markdown doctest
edited Oct 15 '18 at 2:07
jww
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
Does anyone know of an open-source project or program for running tests against code examples in README.md?
A perennial problem, my documentation tends to drift out of date with the code. For example, a code snippet in the README.md will no longer work with the current version, and isn't caught until a new developer on-boards onto the project. Is it possible to include README.md code snippets in my test suite?
For example, the usage of say.nancat with sample params:
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
The program would initialize an environment with the '#' (not shown in README.md because the context is assumed), run the '>' line and pass/fail based on the next line. Simular to doctests in python.
Loads of people have the problem of keeping README.md and other docs current with the code, so I was hoping there was an off-the-shelf solution. I've looked (DuckDuckGo) to no avail.
node.js github markdown doctest
node.js github markdown doctest
edited Oct 15 '18 at 2:07
jww
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
edited Oct 15 '18 at 2:07
jww
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
edited Oct 15 '18 at 2:07
jww
53.6k40231508
edited Oct 15 '18 at 2:07
jww
53.6k40231508
edited Oct 15 '18 at 2:07
jww
53.6k40231508
53.6k40231508
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
asked Sep 27 '18 at 3:21
rmharrisonrmharrison
1,213724
1,213724
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
May be byexample is what you are looking for.
It is a tool to run the snippets of code (aka examples) in a text file and check their outputs. It is like Python's doctests but it works for Javascript, Ruby, Python and others (even for C and C++).
The Javascript examples can be written in a README.md like:
```javascript
1 + 2
out:
3
```
or like:
```javascript
> 1 + 2
3
```
Then, you run them from the command line:
$ byexample -l javascript README.md
[PASS] Pass: 2 Fail: 0 Skip: 0
And that's it. The full documentation of the tool can be found here and here, and the particular comments and limitations for Javascript is here.
Disclaimer: I'm the author of byexample and I created it for the same reason that rmharrison wrote in his question.
Like him, my documentation was "out of sync" in time to time and the only way to notice that was running the examples by hand. For that reason I created this tool to automatically check and validate the docs.
It is really useful to me; I really hope that it would be useful to others.
answered Oct 15 '18 at 1:53
eldipaeldipa
262
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
add a comment |
This possibly should be achieved in the opposite way. Examples should exist as files that can be linted and tested. Their contents can be injected into README.md on documentation build with any template engine.
E.g. custom includeJs helper function can be defined to render
{{ includeJs('foo.js') }}
to respective Markdown:
**foo.js**
```javascript
/* foo.js contents */
```
Depending on how much snippets have in common, the documentation could possibly be parsed first to uniformly generate files from existing snippets.
E.g.
```
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
```
could be transformed into
// grumpy-is-best.js
const say = require('say')
say.nancat('grumpy is best')
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52528952%2frunning-tests-against-code-examples-in-a-readme-md%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
May be byexample is what you are looking for.
It is a tool to run the snippets of code (aka examples) in a text file and check their outputs. It is like Python's doctests but it works for Javascript, Ruby, Python and others (even for C and C++).
The Javascript examples can be written in a README.md like:
```javascript
1 + 2
out:
3
```
or like:
```javascript
> 1 + 2
3
```
Then, you run them from the command line:
$ byexample -l javascript README.md
[PASS] Pass: 2 Fail: 0 Skip: 0
And that's it. The full documentation of the tool can be found here and here, and the particular comments and limitations for Javascript is here.
Disclaimer: I'm the author of byexample and I created it for the same reason that rmharrison wrote in his question.
Like him, my documentation was "out of sync" in time to time and the only way to notice that was running the examples by hand. For that reason I created this tool to automatically check and validate the docs.
It is really useful to me; I really hope that it would be useful to others.
answered Oct 15 '18 at 1:53
eldipaeldipa
262
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
add a comment |
May be byexample is what you are looking for.
It is a tool to run the snippets of code (aka examples) in a text file and check their outputs. It is like Python's doctests but it works for Javascript, Ruby, Python and others (even for C and C++).
The Javascript examples can be written in a README.md like:
```javascript
1 + 2
out:
3
```
or like:
```javascript
> 1 + 2
3
```
Then, you run them from the command line:
$ byexample -l javascript README.md
[PASS] Pass: 2 Fail: 0 Skip: 0
And that's it. The full documentation of the tool can be found here and here, and the particular comments and limitations for Javascript is here.
Disclaimer: I'm the author of byexample and I created it for the same reason that rmharrison wrote in his question.
Like him, my documentation was "out of sync" in time to time and the only way to notice that was running the examples by hand. For that reason I created this tool to automatically check and validate the docs.
It is really useful to me; I really hope that it would be useful to others.
answered Oct 15 '18 at 1:53
eldipaeldipa
262
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
add a comment |
May be byexample is what you are looking for.
It is a tool to run the snippets of code (aka examples) in a text file and check their outputs. It is like Python's doctests but it works for Javascript, Ruby, Python and others (even for C and C++).
The Javascript examples can be written in a README.md like:
```javascript
1 + 2
out:
3
```
or like:
```javascript
> 1 + 2
3
```
Then, you run them from the command line:
$ byexample -l javascript README.md
[PASS] Pass: 2 Fail: 0 Skip: 0
And that's it. The full documentation of the tool can be found here and here, and the particular comments and limitations for Javascript is here.
Disclaimer: I'm the author of byexample and I created it for the same reason that rmharrison wrote in his question.
Like him, my documentation was "out of sync" in time to time and the only way to notice that was running the examples by hand. For that reason I created this tool to automatically check and validate the docs.
It is really useful to me; I really hope that it would be useful to others.
answered Oct 15 '18 at 1:53
eldipaeldipa
262
May be byexample is what you are looking for.
It is a tool to run the snippets of code (aka examples) in a text file and check their outputs. It is like Python's doctests but it works for Javascript, Ruby, Python and others (even for C and C++).
The Javascript examples can be written in a README.md like:
```javascript
1 + 2
out:
3
```
or like:
```javascript
> 1 + 2
3
```
Then, you run them from the command line:
$ byexample -l javascript README.md
[PASS] Pass: 2 Fail: 0 Skip: 0
And that's it. The full documentation of the tool can be found here and here, and the particular comments and limitations for Javascript is here.
Disclaimer: I'm the author of byexample and I created it for the same reason that rmharrison wrote in his question.
Like him, my documentation was "out of sync" in time to time and the only way to notice that was running the examples by hand. For that reason I created this tool to automatically check and validate the docs.
It is really useful to me; I really hope that it would be useful to others.
answered Oct 15 '18 at 1:53
eldipaeldipa
262
edited 2 days ago
answered Oct 15 '18 at 1:53
eldipaeldipa
262
answered Oct 15 '18 at 1:53
eldipaeldipa
262
answered Oct 15 '18 at 1:53
eldipaeldipa
262
262
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
add a comment |
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
Thank you! This is exactly what I was looking for.
– rmharrison
Oct 20 '18 at 5:35
add a comment |
This possibly should be achieved in the opposite way. Examples should exist as files that can be linted and tested. Their contents can be injected into README.md on documentation build with any template engine.
E.g. custom includeJs helper function can be defined to render
{{ includeJs('foo.js') }}
to respective Markdown:
**foo.js**
```javascript
/* foo.js contents */
```
Depending on how much snippets have in common, the documentation could possibly be parsed first to uniformly generate files from existing snippets.
E.g.
```
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
```
could be transformed into
// grumpy-is-best.js
const say = require('say')
say.nancat('grumpy is best')
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
add a comment |
This possibly should be achieved in the opposite way. Examples should exist as files that can be linted and tested. Their contents can be injected into README.md on documentation build with any template engine.
E.g. custom includeJs helper function can be defined to render
{{ includeJs('foo.js') }}
to respective Markdown:
**foo.js**
```javascript
/* foo.js contents */
```
Depending on how much snippets have in common, the documentation could possibly be parsed first to uniformly generate files from existing snippets.
E.g.
```
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
```
could be transformed into
// grumpy-is-best.js
const say = require('say')
say.nancat('grumpy is best')
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
add a comment |
This possibly should be achieved in the opposite way. Examples should exist as files that can be linted and tested. Their contents can be injected into README.md on documentation build with any template engine.
E.g. custom includeJs helper function can be defined to render
{{ includeJs('foo.js') }}
to respective Markdown:
**foo.js**
```javascript
/* foo.js contents */
```
Depending on how much snippets have in common, the documentation could possibly be parsed first to uniformly generate files from existing snippets.
E.g.
```
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
```
could be transformed into
// grumpy-is-best.js
const say = require('say')
say.nancat('grumpy is best')
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
This possibly should be achieved in the opposite way. Examples should exist as files that can be linted and tested. Their contents can be injected into README.md on documentation build with any template engine.
E.g. custom includeJs helper function can be defined to render
{{ includeJs('foo.js') }}
to respective Markdown:
**foo.js**
```javascript
/* foo.js contents */
```
Depending on how much snippets have in common, the documentation could possibly be parsed first to uniformly generate files from existing snippets.
E.g.
```
# $ node
> const say = require('say')
> say.nancat('grumpy is best')
'grumpy is best'
```
could be transformed into
// grumpy-is-best.js
const say = require('say')
say.nancat('grumpy is best')
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
edited Sep 27 '18 at 8:02
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
answered Sep 27 '18 at 7:53
estusestus
75.6k22111228
75.6k22111228
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
add a comment |
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
Thanks for your kind response. So to your knowledge there isn't an off-the-shelf tool; rather, I must roll my own documentation build step with injection from snippets?
– rmharrison
Oct 11 '18 at 18:21
1
1
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
You're welcome. Yes, I would go with doc build step. I'm not aware of such tool and would be surprised if it exists because this would be very odd and complicated approach. It's like writing an application in rich text editor.
– estus
Oct 11 '18 at 18:34
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52528952%2frunning-tests-against-code-examples-in-a-readme-md%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
