Mixing
Need to push jenkins log files to Amazon S3 bucket
I would like to move jenkins logs to s3 bucket.
Can some one suggest how to move the log files.
Need to move console logs from aws instance.
Should I need to write script or the plugin is enough to push log files.
For every last 5 jobs it should move the files to s3.
Based on date, time, build number.
linux shell jenkins amazon-s3 amazon-ec2
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
add a comment |
I would like to move jenkins logs to s3 bucket.
Can some one suggest how to move the log files.
Need to move console logs from aws instance.
Should I need to write script or the plugin is enough to push log files.
For every last 5 jobs it should move the files to s3.
Based on date, time, build number.
linux shell jenkins amazon-s3 amazon-ec2
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
add a comment |
I would like to move jenkins logs to s3 bucket.
Can some one suggest how to move the log files.
Need to move console logs from aws instance.
Should I need to write script or the plugin is enough to push log files.
For every last 5 jobs it should move the files to s3.
Based on date, time, build number.
linux shell jenkins amazon-s3 amazon-ec2
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
I would like to move jenkins logs to s3 bucket.
Can some one suggest how to move the log files.
Need to move console logs from aws instance.
Should I need to write script or the plugin is enough to push log files.
For every last 5 jobs it should move the files to s3.
Based on date, time, build number.
linux shell jenkins amazon-s3 amazon-ec2
linux shell jenkins amazon-s3 amazon-ec2
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
edited Nov 21 '18 at 21:50
K.Dᴀᴠɪs
7,250112439
7,250112439
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
asked Nov 21 '18 at 15:44
Sandeep muthyapuSandeep muthyapu
63
63
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
I suggest to you Script instead of mounting If mounting is not working due to any reason you jobs will start failing Using python script you can directly upload the logs to s3. It will be easier & reliable
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
add a comment |
You can create S3 bucket and mount it to EC2 instance. Detailed information about how to do that you can find HERE.
Once you mounted S3 bucket to your instance you need to say Jenkins that you want to store logs in that mount
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
add a comment |
You first need to determine if your Jenkins server has access to S3 Bucket (as it's unclear in your question)
I am assuming that Jenkins server can access your S3 Bucket (the setup would generally involve the following steps):
- Your Jenkins instance is running on EC2 server and can access S3 either by
- Using a pre-configured role attached to EC2 instance
- Accessing S3 Programmatically
- S3 bucket is correctly configured (Permissions, Policies)
Once the setup is in place, you can use aws-cli to upload files into S3
S3 CLI Reference
Feel free to use Bash/Python/or any other language to wrote the script.
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53415693%2fneed-to-push-jenkins-log-files-to-amazon-s3-bucket%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
I suggest to you Script instead of mounting If mounting is not working due to any reason you jobs will start failing Using python script you can directly upload the logs to s3. It will be easier & reliable
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
add a comment |
I suggest to you Script instead of mounting If mounting is not working due to any reason you jobs will start failing Using python script you can directly upload the logs to s3. It will be easier & reliable
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
add a comment |
I suggest to you Script instead of mounting If mounting is not working due to any reason you jobs will start failing Using python script you can directly upload the logs to s3. It will be easier & reliable
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
I suggest to you Script instead of mounting If mounting is not working due to any reason you jobs will start failing Using python script you can directly upload the logs to s3. It will be easier & reliable
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
answered Nov 21 '18 at 22:43
AnmolNagpalAnmolNagpal
1337
1337
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
add a comment |
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
can you please share the script @anmol Nagpal
– Sandeep muthyapu
Nov 22 '18 at 2:16
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
I'm sorry I don't have a script. But let me try if I can get something for you
– AnmolNagpal
Nov 22 '18 at 10:43
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
Check this one might help github.com/vangheem/sync-ftp-to-s3/blob/master/…
– AnmolNagpal
Nov 22 '18 at 10:44
add a comment |
You can create S3 bucket and mount it to EC2 instance. Detailed information about how to do that you can find HERE.
Once you mounted S3 bucket to your instance you need to say Jenkins that you want to store logs in that mount
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
add a comment |
You can create S3 bucket and mount it to EC2 instance. Detailed information about how to do that you can find HERE.
Once you mounted S3 bucket to your instance you need to say Jenkins that you want to store logs in that mount
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
add a comment |
You can create S3 bucket and mount it to EC2 instance. Detailed information about how to do that you can find HERE.
Once you mounted S3 bucket to your instance you need to say Jenkins that you want to store logs in that mount
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
You can create S3 bucket and mount it to EC2 instance. Detailed information about how to do that you can find HERE.
Once you mounted S3 bucket to your instance you need to say Jenkins that you want to store logs in that mount
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
edited Nov 22 '18 at 10:09
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
answered Nov 21 '18 at 15:54
Roman BanakhRoman Banakh
345
345
add a comment |
add a comment |
You first need to determine if your Jenkins server has access to S3 Bucket (as it's unclear in your question)
I am assuming that Jenkins server can access your S3 Bucket (the setup would generally involve the following steps):
- Your Jenkins instance is running on EC2 server and can access S3 either by
- Using a pre-configured role attached to EC2 instance
- Accessing S3 Programmatically
- S3 bucket is correctly configured (Permissions, Policies)
Once the setup is in place, you can use aws-cli to upload files into S3
S3 CLI Reference
Feel free to use Bash/Python/or any other language to wrote the script.
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
add a comment |
You first need to determine if your Jenkins server has access to S3 Bucket (as it's unclear in your question)
I am assuming that Jenkins server can access your S3 Bucket (the setup would generally involve the following steps):
- Your Jenkins instance is running on EC2 server and can access S3 either by
- Using a pre-configured role attached to EC2 instance
- Accessing S3 Programmatically
- S3 bucket is correctly configured (Permissions, Policies)
Once the setup is in place, you can use aws-cli to upload files into S3
S3 CLI Reference
Feel free to use Bash/Python/or any other language to wrote the script.
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
add a comment |
You first need to determine if your Jenkins server has access to S3 Bucket (as it's unclear in your question)
I am assuming that Jenkins server can access your S3 Bucket (the setup would generally involve the following steps):
- Your Jenkins instance is running on EC2 server and can access S3 either by
- Using a pre-configured role attached to EC2 instance
- Accessing S3 Programmatically
- S3 bucket is correctly configured (Permissions, Policies)
Once the setup is in place, you can use aws-cli to upload files into S3
S3 CLI Reference
Feel free to use Bash/Python/or any other language to wrote the script.
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
You first need to determine if your Jenkins server has access to S3 Bucket (as it's unclear in your question)
I am assuming that Jenkins server can access your S3 Bucket (the setup would generally involve the following steps):
- Your Jenkins instance is running on EC2 server and can access S3 either by
- Using a pre-configured role attached to EC2 instance
- Accessing S3 Programmatically
- S3 bucket is correctly configured (Permissions, Policies)
Once the setup is in place, you can use aws-cli to upload files into S3
S3 CLI Reference
Feel free to use Bash/Python/or any other language to wrote the script.
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
answered Nov 22 '18 at 10:44
ArpitArpit
5,57553164
5,57553164
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53415693%2fneed-to-push-jenkins-log-files-to-amazon-s3-bucket%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
