Problem building a commandline in tcsh to be executed (by lsf), problems with vars and quotes

I am using tcsh (contract required, cannot change to bash etc), but am having a problem building up a command based on various conditions for different pieces.

Some names changed to protect the innocent...

If new or old program name, is really chosen earlier on by a preprocessor, and is hardcoded by the time this shell script gets run:

set myCMDline = newProgName

set myCMDlineTmpFile = "/tmp/myCMDlineTmpScriptFile.csh"

set bsubQname = "typical"



set bsubResources = "span[hosts=1]"

set myCMDline  = "bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline"



($myCMDline)

Now, I have tried several variations of the above, all not working for some reason or another. The closest I think I get is a complaint about mismatched double-quotes, even when backspacing them.

When I do an echo of $myCMDline, then that looks OK, but the execution of same must somehow be different...

set bsubResources = '"span[hosts=1]"'    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R $bsubResources $myCMDline"

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = 'bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline'

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R '$bsubResources' $myCMDline"

etc.

I have also tried dumping to a separate temp script file to source, but that contains the $variable names, not resolved equivalents as I would prefer, as I am doing set, not setenv, and prefer not to put these into shell vars.

First I could not echo the "#!/bin/csh -f" line, it seems to try and execute that rather than echo redirected into the temp script file, and dies.

rm -f $myCMDlineTmpFile

echo "#!/bin/csh -f  > $myCMDlineTmpFile

echo "$myCMDline" >> $myCMDlineTmpFile

($myCMDlineTmpFile)

Then I tried multi-line echo, which is where I am seeing the local variable names go into the file rather than their contents:

/bin/cat > $myCMDlineTmpFile <<EOF

#!/bin/csh -f

$myCMDline



EOF



source $myCMDlineTmpFile

And then I am trying to instead use eval:

eval `echo "$myCMDline &" `

with and without the backticks etc, but complains about unknown variables for the queue name, resources etc.

Adding this echo always looks like what I want to be the commandline, between the >>> and <<<

echo "DEBUG - myCMDline= >>>$myCMDline<<<"

Please help me solve this puzzle...

asked Nov 20 '18 at 22:24

billt

1

You might want to set your variables like var=... not var = ...

– l'L'l
Nov 20 '18 at 22:27

@l'L'l — in a C shell script, you can use set var = value in those, and the spaces around the = are allowed (in tcsh 6.14.00 as found on an antique RHEL 5 Linux). Coming from a Bourne shell background, that seems unintuitive to me, but it is how it works.

– Jonathan Leffler
Nov 20 '18 at 22:51

I don't like having to say it, but maybe you need eval $myCMDline where you currently have ($myCMDline). Another possibility is to write the script you want to execute to a (temporary) file and then run the shell to execute that file. Be cautious about eval when the user had any say in the input you evaluate; it is generally a huge security risk. (Beware: I use tcsh or csh only when forced to do so; I generally use setenv SHELL /bin/bash; exec bash -l as the contents of my .login file, for any shell that isn't a C shell.)

– Jonathan Leffler
Nov 20 '18 at 23:04

@Jonathan - Ive already tried eval and writing to a separate file to run, but am having problems with those as well. Variables are not resolving, the excution is being done using varables that I had intended to have been resolved, and are not shell env vars to be resolved at execution of the commandline. I tried making them shell env vars as well, but that still suffered from unmatched double quotes or other issues.

– billt
Nov 20 '18 at 23:13

Is there a good reason why you need the whole command in a single string? Why not simply use bsub -q "$bsubQname" -n 8 -R "$bsubResources" "$myCMDline" (omitting the incremental assignment to myCMDline)? You can put that in a sub-shell if you like. Doing it all in one string complicates life enormously.

– Jonathan Leffler
Nov 20 '18 at 23:19

|
show 2 more comments

I am using tcsh (contract required, cannot change to bash etc), but am having a problem building up a command based on various conditions for different pieces.

Some names changed to protect the innocent...

If new or old program name, is really chosen earlier on by a preprocessor, and is hardcoded by the time this shell script gets run:

set myCMDline = newProgName

set myCMDlineTmpFile = "/tmp/myCMDlineTmpScriptFile.csh"

set bsubQname = "typical"



set bsubResources = "span[hosts=1]"

set myCMDline  = "bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline"



($myCMDline)

Now, I have tried several variations of the above, all not working for some reason or another. The closest I think I get is a complaint about mismatched double-quotes, even when backspacing them.

When I do an echo of $myCMDline, then that looks OK, but the execution of same must somehow be different...

set bsubResources = '"span[hosts=1]"'    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R $bsubResources $myCMDline"

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = 'bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline'

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R '$bsubResources' $myCMDline"

etc.

First I could not echo the "#!/bin/csh -f" line, it seems to try and execute that rather than echo redirected into the temp script file, and dies.

rm -f $myCMDlineTmpFile

echo "#!/bin/csh -f  > $myCMDlineTmpFile

echo "$myCMDline" >> $myCMDlineTmpFile

($myCMDlineTmpFile)

Then I tried multi-line echo, which is where I am seeing the local variable names go into the file rather than their contents:

/bin/cat > $myCMDlineTmpFile <<EOF

#!/bin/csh -f

$myCMDline



EOF



source $myCMDlineTmpFile

And then I am trying to instead use eval:

eval `echo "$myCMDline &" `

with and without the backticks etc, but complains about unknown variables for the queue name, resources etc.

Adding this echo always looks like what I want to be the commandline, between the >>> and <<<

echo "DEBUG - myCMDline= >>>$myCMDline<<<"

Please help me solve this puzzle...

asked Nov 20 '18 at 22:24

billt

1

You might want to set your variables like var=... not var = ...

– l'L'l
Nov 20 '18 at 22:27

@l'L'l — in a C shell script, you can use set var = value in those, and the spaces around the = are allowed (in tcsh 6.14.00 as found on an antique RHEL 5 Linux). Coming from a Bourne shell background, that seems unintuitive to me, but it is how it works.

– Jonathan Leffler
Nov 20 '18 at 22:51

I don't like having to say it, but maybe you need eval $myCMDline where you currently have ($myCMDline). Another possibility is to write the script you want to execute to a (temporary) file and then run the shell to execute that file. Be cautious about eval when the user had any say in the input you evaluate; it is generally a huge security risk. (Beware: I use tcsh or csh only when forced to do so; I generally use setenv SHELL /bin/bash; exec bash -l as the contents of my .login file, for any shell that isn't a C shell.)

– Jonathan Leffler
Nov 20 '18 at 23:04

@Jonathan - Ive already tried eval and writing to a separate file to run, but am having problems with those as well. Variables are not resolving, the excution is being done using varables that I had intended to have been resolved, and are not shell env vars to be resolved at execution of the commandline. I tried making them shell env vars as well, but that still suffered from unmatched double quotes or other issues.

– billt
Nov 20 '18 at 23:13

Is there a good reason why you need the whole command in a single string? Why not simply use bsub -q "$bsubQname" -n 8 -R "$bsubResources" "$myCMDline" (omitting the incremental assignment to myCMDline)? You can put that in a sub-shell if you like. Doing it all in one string complicates life enormously.

– Jonathan Leffler
Nov 20 '18 at 23:19

|
show 2 more comments

I am using tcsh (contract required, cannot change to bash etc), but am having a problem building up a command based on various conditions for different pieces.

Some names changed to protect the innocent...

If new or old program name, is really chosen earlier on by a preprocessor, and is hardcoded by the time this shell script gets run:

set myCMDline = newProgName

set myCMDlineTmpFile = "/tmp/myCMDlineTmpScriptFile.csh"

set bsubQname = "typical"



set bsubResources = "span[hosts=1]"

set myCMDline  = "bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline"



($myCMDline)

Now, I have tried several variations of the above, all not working for some reason or another. The closest I think I get is a complaint about mismatched double-quotes, even when backspacing them.

When I do an echo of $myCMDline, then that looks OK, but the execution of same must somehow be different...

set bsubResources = '"span[hosts=1]"'    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R $bsubResources $myCMDline"

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = 'bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline'

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R '$bsubResources' $myCMDline"

etc.

First I could not echo the "#!/bin/csh -f" line, it seems to try and execute that rather than echo redirected into the temp script file, and dies.

rm -f $myCMDlineTmpFile

echo "#!/bin/csh -f  > $myCMDlineTmpFile

echo "$myCMDline" >> $myCMDlineTmpFile

($myCMDlineTmpFile)

Then I tried multi-line echo, which is where I am seeing the local variable names go into the file rather than their contents:

/bin/cat > $myCMDlineTmpFile <<EOF

#!/bin/csh -f

$myCMDline



EOF



source $myCMDlineTmpFile

And then I am trying to instead use eval:

eval `echo "$myCMDline &" `

with and without the backticks etc, but complains about unknown variables for the queue name, resources etc.

Adding this echo always looks like what I want to be the commandline, between the >>> and <<<

echo "DEBUG - myCMDline= >>>$myCMDline<<<"

Please help me solve this puzzle...

asked Nov 20 '18 at 22:24

billt

I am using tcsh (contract required, cannot change to bash etc), but am having a problem building up a command based on various conditions for different pieces.

Some names changed to protect the innocent...

If new or old program name, is really chosen earlier on by a preprocessor, and is hardcoded by the time this shell script gets run:

set myCMDline = newProgName

set myCMDlineTmpFile = "/tmp/myCMDlineTmpScriptFile.csh"

set bsubQname = "typical"



set bsubResources = "span[hosts=1]"

set myCMDline  = "bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline"



($myCMDline)

Now, I have tried several variations of the above, all not working for some reason or another. The closest I think I get is a complaint about mismatched double-quotes, even when backspacing them.

When I do an echo of $myCMDline, then that looks OK, but the execution of same must somehow be different...

set bsubResources = '"span[hosts=1]"'    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R $bsubResources $myCMDline"

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = 'bsub -q $bsubQname -n 8 -R "$bsubResources" $myCMDline'

set bsubResources = "span[hosts=1]"    #double-quotes inside, single-quotes outside

set myCMDline  = "bsub -q $bsubQname -n 8 -R '$bsubResources' $myCMDline"

etc.

First I could not echo the "#!/bin/csh -f" line, it seems to try and execute that rather than echo redirected into the temp script file, and dies.

rm -f $myCMDlineTmpFile

echo "#!/bin/csh -f  > $myCMDlineTmpFile

echo "$myCMDline" >> $myCMDlineTmpFile

($myCMDlineTmpFile)

Then I tried multi-line echo, which is where I am seeing the local variable names go into the file rather than their contents:

/bin/cat > $myCMDlineTmpFile <<EOF

#!/bin/csh -f

$myCMDline



EOF



source $myCMDlineTmpFile

And then I am trying to instead use eval:

eval `echo "$myCMDline &" `

with and without the backticks etc, but complains about unknown variables for the queue name, resources etc.

Adding this echo always looks like what I want to be the commandline, between the >>> and <<<

echo "DEBUG - myCMDline= >>>$myCMDline<<<"

Please help me solve this puzzle...

shell variables command-line csh tcsh

asked Nov 20 '18 at 22:24

billt

asked Nov 20 '18 at 22:24

billt

asked Nov 20 '18 at 22:24

billt

asked Nov 20 '18 at 22:24

billt

asked Nov 20 '18 at 22:24

billt

1

You might want to set your variables like var=... not var = ...

– l'L'l
Nov 20 '18 at 22:27

@l'L'l — in a C shell script, you can use set var = value in those, and the spaces around the = are allowed (in tcsh 6.14.00 as found on an antique RHEL 5 Linux). Coming from a Bourne shell background, that seems unintuitive to me, but it is how it works.

– Jonathan Leffler
Nov 20 '18 at 22:51

I don't like having to say it, but maybe you need eval $myCMDline where you currently have ($myCMDline). Another possibility is to write the script you want to execute to a (temporary) file and then run the shell to execute that file. Be cautious about eval when the user had any say in the input you evaluate; it is generally a huge security risk. (Beware: I use tcsh or csh only when forced to do so; I generally use setenv SHELL /bin/bash; exec bash -l as the contents of my .login file, for any shell that isn't a C shell.)

– Jonathan Leffler
Nov 20 '18 at 23:04

@Jonathan - Ive already tried eval and writing to a separate file to run, but am having problems with those as well. Variables are not resolving, the excution is being done using varables that I had intended to have been resolved, and are not shell env vars to be resolved at execution of the commandline. I tried making them shell env vars as well, but that still suffered from unmatched double quotes or other issues.

– billt
Nov 20 '18 at 23:13

Is there a good reason why you need the whole command in a single string? Why not simply use bsub -q "$bsubQname" -n 8 -R "$bsubResources" "$myCMDline" (omitting the incremental assignment to myCMDline)? You can put that in a sub-shell if you like. Doing it all in one string complicates life enormously.

– Jonathan Leffler
Nov 20 '18 at 23:19

|
show 2 more comments

1

You might want to set your variables like var=... not var = ...

– l'L'l
Nov 20 '18 at 22:27

@l'L'l — in a C shell script, you can use set var = value in those, and the spaces around the = are allowed (in tcsh 6.14.00 as found on an antique RHEL 5 Linux). Coming from a Bourne shell background, that seems unintuitive to me, but it is how it works.

– Jonathan Leffler
Nov 20 '18 at 22:51

I don't like having to say it, but maybe you need eval $myCMDline where you currently have ($myCMDline). Another possibility is to write the script you want to execute to a (temporary) file and then run the shell to execute that file. Be cautious about eval when the user had any say in the input you evaluate; it is generally a huge security risk. (Beware: I use tcsh or csh only when forced to do so; I generally use setenv SHELL /bin/bash; exec bash -l as the contents of my .login file, for any shell that isn't a C shell.)

– Jonathan Leffler
Nov 20 '18 at 23:04

@Jonathan - Ive already tried eval and writing to a separate file to run, but am having problems with those as well. Variables are not resolving, the excution is being done using varables that I had intended to have been resolved, and are not shell env vars to be resolved at execution of the commandline. I tried making them shell env vars as well, but that still suffered from unmatched double quotes or other issues.

– billt
Nov 20 '18 at 23:13

Is there a good reason why you need the whole command in a single string? Why not simply use bsub -q "$bsubQname" -n 8 -R "$bsubResources" "$myCMDline" (omitting the incremental assignment to myCMDline)? You can put that in a sub-shell if you like. Doing it all in one string complicates life enormously.

– Jonathan Leffler
Nov 20 '18 at 23:19

You might want to set your variables like var=... not var = ...

– l'L'l
Nov 20 '18 at 22:27

@l'L'l — in a C shell script, you can use set var = value in those, and the spaces around the = are allowed (in tcsh 6.14.00 as found on an antique RHEL 5 Linux). Coming from a Bourne shell background, that seems unintuitive to me, but it is how it works.

– Jonathan Leffler
Nov 20 '18 at 22:51

I don't like having to say it, but maybe you need eval $myCMDline where you currently have ($myCMDline). Another possibility is to write the script you want to execute to a (temporary) file and then run the shell to execute that file. Be cautious about eval when the user had any say in the input you evaluate; it is generally a huge security risk. (Beware: I use tcsh or csh only when forced to do so; I generally use setenv SHELL /bin/bash; exec bash -l as the contents of my .login file, for any shell that isn't a C shell.)

– Jonathan Leffler
Nov 20 '18 at 23:04

@Jonathan - Ive already tried eval and writing to a separate file to run, but am having problems with those as well. Variables are not resolving, the excution is being done using varables that I had intended to have been resolved, and are not shell env vars to be resolved at execution of the commandline. I tried making them shell env vars as well, but that still suffered from unmatched double quotes or other issues.

– billt
Nov 20 '18 at 23:13

Is there a good reason why you need the whole command in a single string? Why not simply use bsub -q "$bsubQname" -n 8 -R "$bsubResources" "$myCMDline" (omitting the incremental assignment to myCMDline)? You can put that in a sub-shell if you like. Doing it all in one string complicates life enormously.

– Jonathan Leffler
Nov 20 '18 at 23:19

|
show 2 more comments

0

active

oldest

votes

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53402505%2fproblem-building-a-commandline-in-tcsh-to-be-executed-by-lsf-problems-with-va%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

0

active

oldest

votes

0

active

oldest

votes

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

Search This Blog

Ufyukyu