Mixing
Splunk forwarder with kubernates in side car pattern
I have created a custom splunkforwarder image.
Image name: vrathore/splunkuniversalforwarder
I have verified that the log is pushing to the server. I am using dummy log present in my host(c/Users/var/log).If I run this docker command:
docker run --name splunkforwarder -d -v /c/Users/var/log://var/log/messages -p 8089:8089 -p 8088:8088 -e SPLUNK_SERVER_HOST=splunk-prodtest-gsp.test.com:9997 -e
FORWARD_HOSTNAME=kubernetes vrathore/splunkuniversalforwarder
Now I wanted to use the same image in kubernetes pod. where 2 container will share their log folder with my splunk forwarder image.
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Kubernetes is new to me. Could someone please help me how can I share the log folder between the containers. Thanks
kubernetes splunk
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
add a comment |
I have created a custom splunkforwarder image.
Image name: vrathore/splunkuniversalforwarder
I have verified that the log is pushing to the server. I am using dummy log present in my host(c/Users/var/log).If I run this docker command:
docker run --name splunkforwarder -d -v /c/Users/var/log://var/log/messages -p 8089:8089 -p 8088:8088 -e SPLUNK_SERVER_HOST=splunk-prodtest-gsp.test.com:9997 -e
FORWARD_HOSTNAME=kubernetes vrathore/splunkuniversalforwarder
Now I wanted to use the same image in kubernetes pod. where 2 container will share their log folder with my splunk forwarder image.
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Kubernetes is new to me. Could someone please help me how can I share the log folder between the containers. Thanks
kubernetes splunk
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
add a comment |
I have created a custom splunkforwarder image.
Image name: vrathore/splunkuniversalforwarder
I have verified that the log is pushing to the server. I am using dummy log present in my host(c/Users/var/log).If I run this docker command:
docker run --name splunkforwarder -d -v /c/Users/var/log://var/log/messages -p 8089:8089 -p 8088:8088 -e SPLUNK_SERVER_HOST=splunk-prodtest-gsp.test.com:9997 -e
FORWARD_HOSTNAME=kubernetes vrathore/splunkuniversalforwarder
Now I wanted to use the same image in kubernetes pod. where 2 container will share their log folder with my splunk forwarder image.
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Kubernetes is new to me. Could someone please help me how can I share the log folder between the containers. Thanks
kubernetes splunk
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
I have created a custom splunkforwarder image.
Image name: vrathore/splunkuniversalforwarder
I have verified that the log is pushing to the server. I am using dummy log present in my host(c/Users/var/log).If I run this docker command:
docker run --name splunkforwarder -d -v /c/Users/var/log://var/log/messages -p 8089:8089 -p 8088:8088 -e SPLUNK_SERVER_HOST=splunk-prodtest-gsp.test.com:9997 -e
FORWARD_HOSTNAME=kubernetes vrathore/splunkuniversalforwarder
Now I wanted to use the same image in kubernetes pod. where 2 container will share their log folder with my splunk forwarder image.
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Kubernetes is new to me. Could someone please help me how can I share the log folder between the containers. Thanks
kubernetes splunk
kubernetes splunk
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
edited Nov 20 '18 at 10:03
Prafull Ladha
2,784320
2,784320
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
asked Nov 20 '18 at 9:36
gamechanger17gamechanger17
1481111
1481111
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You need to define an emptyDir type volume and attach it to both containers. Assuming that the logs from the app are under /var/log/myapp/ (I have added the second container as well)
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
- name: uf
image: vrathore/splunkuniversalforwarder
...
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
volumes:
- name: logs
emptyDir: {}
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Also, I would recommend looking for an alternative solution, with Collectord and Monitoring Kubernetes/OpenShift you can tell Collectord where to look for logs and you don't need to run a sidecar container https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#application-logs, just one Collectord daemon will do the work.
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53390039%2fsplunk-forwarder-with-kubernates-in-side-car-pattern%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You need to define an emptyDir type volume and attach it to both containers. Assuming that the logs from the app are under /var/log/myapp/ (I have added the second container as well)
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
- name: uf
image: vrathore/splunkuniversalforwarder
...
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
volumes:
- name: logs
emptyDir: {}
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Also, I would recommend looking for an alternative solution, with Collectord and Monitoring Kubernetes/OpenShift you can tell Collectord where to look for logs and you don't need to run a sidecar container https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#application-logs, just one Collectord daemon will do the work.
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
add a comment |
You need to define an emptyDir type volume and attach it to both containers. Assuming that the logs from the app are under /var/log/myapp/ (I have added the second container as well)
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
- name: uf
image: vrathore/splunkuniversalforwarder
...
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
volumes:
- name: logs
emptyDir: {}
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Also, I would recommend looking for an alternative solution, with Collectord and Monitoring Kubernetes/OpenShift you can tell Collectord where to look for logs and you don't need to run a sidecar container https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#application-logs, just one Collectord daemon will do the work.
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
add a comment |
You need to define an emptyDir type volume and attach it to both containers. Assuming that the logs from the app are under /var/log/myapp/ (I have added the second container as well)
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
- name: uf
image: vrathore/splunkuniversalforwarder
...
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
volumes:
- name: logs
emptyDir: {}
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Also, I would recommend looking for an alternative solution, with Collectord and Monitoring Kubernetes/OpenShift you can tell Collectord where to look for logs and you don't need to run a sidecar container https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#application-logs, just one Collectord daemon will do the work.
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
You need to define an emptyDir type volume and attach it to both containers. Assuming that the logs from the app are under /var/log/myapp/ (I have added the second container as well)
spec:
revisionHistoryLimit: 5
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 10%
maxSurge: 10%
replicas: 1
template:
metadata:
name: %APP_FULL_NAME%-pod
labels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
component: app-kube-pod-object
spec:
containers:
- name: %APP_FULL_NAME%-service
image: %DOCKER_IMAGE%
imagePullPolicy: Always
envFrom:
- configMapRef:
name: %APP_CONFIG_MAP%
command: ["catalina.sh", "run"]
ports:
- containerPort: 8080
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
- name: uf
image: vrathore/splunkuniversalforwarder
...
volumeMounts:
- name: logs
mountPath: /var/log/myapp/
imagePullSecrets:
- name: %DOCKER_REPO_REGKEY%
volumes:
- name: logs
emptyDir: {}
selector:
matchLabels:
appname: %APP_FULL_NAME%
stage: %APP_ENV%
Also, I would recommend looking for an alternative solution, with Collectord and Monitoring Kubernetes/OpenShift you can tell Collectord where to look for logs and you don't need to run a sidecar container https://www.outcoldsolutions.com/docs/monitoring-kubernetes/v5/annotations/#application-logs, just one Collectord daemon will do the work.
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
answered Nov 20 '18 at 14:02
outcoldmanoutcoldman
8,53621626
8,53621626
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
add a comment |
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
Thank you for replying. But what if I need to make a tweak like container 1 have logs in /usr/var/log/myapp/tomcat but I want that logs in 2nd container in under /var/log/message.
– gamechanger17
Nov 22 '18 at 6:54
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
You can mount them to a different path. That is fine.
– outcoldman
Nov 22 '18 at 19:20
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53390039%2fsplunk-forwarder-with-kubernates-in-side-car-pattern%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
