Shell script that filters command output and saves it in Json formated list

never worked with shell scripts before,but i need to in my current task.

So i have to run a command that returns output like this:

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1  

    awdafawe23413f13a3r3r refs/heads/SomeInfo2 

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3 

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4

And i need to loop over every line of output get only the "SomeInfo" part and write it to a file in a format like this:

    ["SomeInfo1","SomeInfo2","SomeInfo3"]

I've tried things like this:

    for i in $(some command); do

      echo $i | cut -f2 -d"heads/" >> text.txt

    done

But i don't know how to format it into an array without using a temporary file.

Sorry if the question is dumb and probably too easy and im sure i can figure it out on my own,but i just don't have the time for it because its just an extra conveniance feature that i personally want to implement.

asked Nov 20 '18 at 15:56

Petar Nikolov

132

add a comment |

never worked with shell scripts before,but i need to in my current task.

So i have to run a command that returns output like this:

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1  

    awdafawe23413f13a3r3r refs/heads/SomeInfo2 

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3 

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4

And i need to loop over every line of output get only the "SomeInfo" part and write it to a file in a format like this:

    ["SomeInfo1","SomeInfo2","SomeInfo3"]

I've tried things like this:

    for i in $(some command); do

      echo $i | cut -f2 -d"heads/" >> text.txt

    done

asked Nov 20 '18 at 15:56

Petar Nikolov

132

add a comment |

never worked with shell scripts before,but i need to in my current task.

So i have to run a command that returns output like this:

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1  

    awdafawe23413f13a3r3r refs/heads/SomeInfo2 

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3 

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4

And i need to loop over every line of output get only the "SomeInfo" part and write it to a file in a format like this:

    ["SomeInfo1","SomeInfo2","SomeInfo3"]

I've tried things like this:

    for i in $(some command); do

      echo $i | cut -f2 -d"heads/" >> text.txt

    done

asked Nov 20 '18 at 15:56

Petar Nikolov

132

never worked with shell scripts before,but i need to in my current task.

So i have to run a command that returns output like this:

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1  

    awdafawe23413f13a3r3r refs/heads/SomeInfo2 

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3 

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4

And i need to loop over every line of output get only the "SomeInfo" part and write it to a file in a format like this:

    ["SomeInfo1","SomeInfo2","SomeInfo3"]

I've tried things like this:

    for i in $(some command); do

      echo $i | cut -f2 -d"heads/" >> text.txt

    done

linux bash shell string-formatting

asked Nov 20 '18 at 15:56

Petar Nikolov

132

asked Nov 20 '18 at 15:56

Petar Nikolov

132

asked Nov 20 '18 at 15:56

Petar Nikolov

132

asked Nov 20 '18 at 15:56

Petar Nikolov

132

asked Nov 20 '18 at 15:56

Petar Nikolov

132

add a comment |

4 Answers
4

active

oldest

votes

Using Perl one-liner

$ cat petar.txt

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1

    awdafawe23413f13a3r3r refs/heads/SomeInfo2

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4



$ perl -ne ' { /.*/(.*)/ and push(@res,""$1"") } END { print "[".join(",",@res)."]n" }' petar.txt

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

add a comment |

Try this

# json_encoder.sh

arr=()

while read line; do

  arr+=("$(basename "$line")")

done

printf "[%s]" $(IFS=,; echo "${arr[*]}")

And then invoke

./your_command | json_encoder.sh

PS. I personally do this kind of data massaging with Vim.

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

add a comment |

While you should rarely ever use a script to format json, in your case you are simply parsing output into a comma-separated line with added end-caps of [...]. You can use bash parameter expansion to avoid spawning any additional subshells to obtain the last field of information in each line as follows:

#!/bin/bash



[ -z "$1" -o ! -r "$1" ] && {   ## validate file given as argument

    printf "error: file doesn't exist or not readable.n" >&2

    exit 1

}



c=0     ## simple flag variable



while read -r line; do                  ## read each line

    if [ "$c" -eq '0' ]; then           ## is flag 0?

        printf "["%s"" "${line##*/}"  ## output ["last"

    else                                ## otherwise

        printf ","%s"" "${line##*/}"  ## output ,"last"

    fi

    c=1         ## set flag 1

done < file     ## redirect file to loop

echo "]"        ## append closing ]

Example Use/Output

Using your given data as the input file, you would get the following:

$ bash script.sh file

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

Look things over and let me know if you have any questions.

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

add a comment |

You can also use awk without any loops I guess:

cat prev_output | awk -v ORS=',' -F'/' '{print "42"$3"42"}' | 

sed 's/^/[/g ; s/,$/]n/g' > new_output



cat new_output

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53396836%2fshell-script-that-filters-command-output-and-saves-it-in-json-formated-list%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

4 Answers
4

active

oldest

votes

4 Answers
4

active

oldest

votes

Using Perl one-liner

$ cat petar.txt

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1

    awdafawe23413f13a3r3r refs/heads/SomeInfo2

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4



$ perl -ne ' { /.*/(.*)/ and push(@res,""$1"") } END { print "[".join(",",@res)."]n" }' petar.txt

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

add a comment |

Using Perl one-liner

$ cat petar.txt

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1

    awdafawe23413f13a3r3r refs/heads/SomeInfo2

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4



$ perl -ne ' { /.*/(.*)/ and push(@res,""$1"") } END { print "[".join(",",@res)."]n" }' petar.txt

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

add a comment |

Using Perl one-liner

$ cat petar.txt

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1

    awdafawe23413f13a3r3r refs/heads/SomeInfo2

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4



$ perl -ne ' { /.*/(.*)/ and push(@res,""$1"") } END { print "[".join(",",@res)."]n" }' petar.txt

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

Using Perl one-liner

$ cat petar.txt

    awd54a7w6ds54awd47awd refs/heads/SomeInfo1

    awdafawe23413f13a3r3r refs/heads/SomeInfo2

    a8wd5a8w5da78d6asawd7 refs/heads/SomeInfo3

    g9reh9wrg69egs7ef987e refs/heads/SomeInfo4



$ perl -ne ' { /.*/(.*)/ and push(@res,""$1"") } END { print "[".join(",",@res)."]n" }' petar.txt

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

answered Nov 20 '18 at 20:27

stack0114106

2,9871417

add a comment |

Try this

# json_encoder.sh

arr=()

while read line; do

  arr+=("$(basename "$line")")

done

printf "[%s]" $(IFS=,; echo "${arr[*]}")

And then invoke

./your_command | json_encoder.sh

PS. I personally do this kind of data massaging with Vim.

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

add a comment |

Try this

# json_encoder.sh

arr=()

while read line; do

  arr+=("$(basename "$line")")

done

printf "[%s]" $(IFS=,; echo "${arr[*]}")

And then invoke

./your_command | json_encoder.sh

PS. I personally do this kind of data massaging with Vim.

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

add a comment |

Try this

# json_encoder.sh

arr=()

while read line; do

  arr+=("$(basename "$line")")

done

printf "[%s]" $(IFS=,; echo "${arr[*]}")

And then invoke

./your_command | json_encoder.sh

PS. I personally do this kind of data massaging with Vim.

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

Try this

# json_encoder.sh

arr=()

while read line; do

  arr+=("$(basename "$line")")

done

printf "[%s]" $(IFS=,; echo "${arr[*]}")

And then invoke

./your_command | json_encoder.sh

PS. I personally do this kind of data massaging with Vim.

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

answered Nov 20 '18 at 16:22

Elias Toivanen

1806

add a comment |

#!/bin/bash



[ -z "$1" -o ! -r "$1" ] && {   ## validate file given as argument

    printf "error: file doesn't exist or not readable.n" >&2

    exit 1

}



c=0     ## simple flag variable



while read -r line; do                  ## read each line

    if [ "$c" -eq '0' ]; then           ## is flag 0?

        printf "["%s"" "${line##*/}"  ## output ["last"

    else                                ## otherwise

        printf ","%s"" "${line##*/}"  ## output ,"last"

    fi

    c=1         ## set flag 1

done < file     ## redirect file to loop

echo "]"        ## append closing ]

Example Use/Output

Using your given data as the input file, you would get the following:

$ bash script.sh file

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

Look things over and let me know if you have any questions.

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

add a comment |

#!/bin/bash



[ -z "$1" -o ! -r "$1" ] && {   ## validate file given as argument

    printf "error: file doesn't exist or not readable.n" >&2

    exit 1

}



c=0     ## simple flag variable



while read -r line; do                  ## read each line

    if [ "$c" -eq '0' ]; then           ## is flag 0?

        printf "["%s"" "${line##*/}"  ## output ["last"

    else                                ## otherwise

        printf ","%s"" "${line##*/}"  ## output ,"last"

    fi

    c=1         ## set flag 1

done < file     ## redirect file to loop

echo "]"        ## append closing ]

Example Use/Output

Using your given data as the input file, you would get the following:

$ bash script.sh file

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

Look things over and let me know if you have any questions.

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

add a comment |

#!/bin/bash



[ -z "$1" -o ! -r "$1" ] && {   ## validate file given as argument

    printf "error: file doesn't exist or not readable.n" >&2

    exit 1

}



c=0     ## simple flag variable



while read -r line; do                  ## read each line

    if [ "$c" -eq '0' ]; then           ## is flag 0?

        printf "["%s"" "${line##*/}"  ## output ["last"

    else                                ## otherwise

        printf ","%s"" "${line##*/}"  ## output ,"last"

    fi

    c=1         ## set flag 1

done < file     ## redirect file to loop

echo "]"        ## append closing ]

Example Use/Output

Using your given data as the input file, you would get the following:

$ bash script.sh file

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

Look things over and let me know if you have any questions.

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

#!/bin/bash



[ -z "$1" -o ! -r "$1" ] && {   ## validate file given as argument

    printf "error: file doesn't exist or not readable.n" >&2

    exit 1

}



c=0     ## simple flag variable



while read -r line; do                  ## read each line

    if [ "$c" -eq '0' ]; then           ## is flag 0?

        printf "["%s"" "${line##*/}"  ## output ["last"

    else                                ## otherwise

        printf ","%s"" "${line##*/}"  ## output ,"last"

    fi

    c=1         ## set flag 1

done < file     ## redirect file to loop

echo "]"        ## append closing ]

Example Use/Output

Using your given data as the input file, you would get the following:

$ bash script.sh file

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

Look things over and let me know if you have any questions.

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

answered Nov 20 '18 at 17:07

David C. Rankin

41.1k32648

add a comment |

You can also use awk without any loops I guess:

cat prev_output | awk -v ORS=',' -F'/' '{print "42"$3"42"}' | 

sed 's/^/[/g ; s/,$/]n/g' > new_output



cat new_output

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

add a comment |

You can also use awk without any loops I guess:

cat prev_output | awk -v ORS=',' -F'/' '{print "42"$3"42"}' | 

sed 's/^/[/g ; s/,$/]n/g' > new_output



cat new_output

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

add a comment |

You can also use awk without any loops I guess:

cat prev_output | awk -v ORS=',' -F'/' '{print "42"$3"42"}' | 

sed 's/^/[/g ; s/,$/]n/g' > new_output



cat new_output

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

You can also use awk without any loops I guess:

cat prev_output | awk -v ORS=',' -F'/' '{print "42"$3"42"}' | 

sed 's/^/[/g ; s/,$/]n/g' > new_output



cat new_output

["SomeInfo1","SomeInfo2","SomeInfo3","SomeInfo4"]

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

edited Nov 20 '18 at 17:13

answered Nov 20 '18 at 17:07

MikeKatz45

1738

answered Nov 20 '18 at 17:07

MikeKatz45

1738

answered Nov 20 '18 at 17:07

MikeKatz45

1738

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu