Mixing
Do typical system call interfaces allow reducing the size of a file (without replacing it with a different...
Is there a way to open() a file and cause it to shrink? One can, of course, open them in append-mode or seek to the end and write to cause them to grow. However, as far as I know, there is no method to shrink a file via typical unix-style system call interfaces.
The only way to do so, as far as I know, is by faking it by creating a new shorter file and rename() it in place of the older one.
I just wanted confirmation, because I saw an answer that implied that it was possible to make file editors that worked directly on a file instead of going through the process of making a new one and renaming it in place.
I've always thought that the file api in libc and unix-style system call interfaces did not allow for the shrinking of files to ease implementation of filesystems and maybe avoid usage patterns that might contribute to fragmentation.
files filesystems
asked Jan 23 at 2:07
JoLJoL
1,146311
add a comment |
Is there a way to open() a file and cause it to shrink? One can, of course, open them in append-mode or seek to the end and write to cause them to grow. However, as far as I know, there is no method to shrink a file via typical unix-style system call interfaces.
The only way to do so, as far as I know, is by faking it by creating a new shorter file and rename() it in place of the older one.
I just wanted confirmation, because I saw an answer that implied that it was possible to make file editors that worked directly on a file instead of going through the process of making a new one and renaming it in place.
I've always thought that the file api in libc and unix-style system call interfaces did not allow for the shrinking of files to ease implementation of filesystems and maybe avoid usage patterns that might contribute to fragmentation.
files filesystems
asked Jan 23 at 2:07
JoLJoL
1,146311
2
Just opening a file withfopenin mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?
– Wyzard
Jan 23 at 7:43
4
Just FYI,open()andopenat()already have a flag for truncating,O_TRUNCso technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that iscommand > file.txt, where file will be truncated if it exists. If you runstraceonbash -c 'true > /dev/null'you'll seeopenat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666)in the output. Of course for variable size of truncation you needtruncate()syscall. Let me know if you want this as an actual answer instead of the comment.
– Sergiy Kolodyazhnyy
Jan 23 at 8:19
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18
add a comment |
Is there a way to open() a file and cause it to shrink? One can, of course, open them in append-mode or seek to the end and write to cause them to grow. However, as far as I know, there is no method to shrink a file via typical unix-style system call interfaces.
The only way to do so, as far as I know, is by faking it by creating a new shorter file and rename() it in place of the older one.
I just wanted confirmation, because I saw an answer that implied that it was possible to make file editors that worked directly on a file instead of going through the process of making a new one and renaming it in place.
I've always thought that the file api in libc and unix-style system call interfaces did not allow for the shrinking of files to ease implementation of filesystems and maybe avoid usage patterns that might contribute to fragmentation.
files filesystems
asked Jan 23 at 2:07
JoLJoL
1,146311
Is there a way to open() a file and cause it to shrink? One can, of course, open them in append-mode or seek to the end and write to cause them to grow. However, as far as I know, there is no method to shrink a file via typical unix-style system call interfaces.
The only way to do so, as far as I know, is by faking it by creating a new shorter file and rename() it in place of the older one.
I just wanted confirmation, because I saw an answer that implied that it was possible to make file editors that worked directly on a file instead of going through the process of making a new one and renaming it in place.
I've always thought that the file api in libc and unix-style system call interfaces did not allow for the shrinking of files to ease implementation of filesystems and maybe avoid usage patterns that might contribute to fragmentation.
files filesystems
files filesystems
asked Jan 23 at 2:07
JoLJoL
1,146311
asked Jan 23 at 2:07
JoLJoL
1,146311
asked Jan 23 at 2:07
JoLJoL
1,146311
asked Jan 23 at 2:07
JoLJoL
1,146311
asked Jan 23 at 2:07
JoLJoL
1,146311
1,146311
2
Just opening a file withfopenin mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?
– Wyzard
Jan 23 at 7:43
4
Just FYI,open()andopenat()already have a flag for truncating,O_TRUNCso technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that iscommand > file.txt, where file will be truncated if it exists. If you runstraceonbash -c 'true > /dev/null'you'll seeopenat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666)in the output. Of course for variable size of truncation you needtruncate()syscall. Let me know if you want this as an actual answer instead of the comment.
– Sergiy Kolodyazhnyy
Jan 23 at 8:19
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18
add a comment |
2
Just opening a file withfopenin mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?
– Wyzard
Jan 23 at 7:43
4
Just FYI,open()andopenat()already have a flag for truncating,O_TRUNCso technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that iscommand > file.txt, where file will be truncated if it exists. If you runstraceonbash -c 'true > /dev/null'you'll seeopenat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666)in the output. Of course for variable size of truncation you needtruncate()syscall. Let me know if you want this as an actual answer instead of the comment.
– Sergiy Kolodyazhnyy
Jan 23 at 8:19
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18
2
2
Just opening a file with
fopen in mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?– Wyzard
Jan 23 at 7:43
Just opening a file with
fopen in mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?– Wyzard
Jan 23 at 7:43
4
4
Just FYI,
open() and openat() already have a flag for truncating, O_TRUNC so technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that is command > file.txt, where file will be truncated if it exists. If you run strace on bash -c 'true > /dev/null' you'll see openat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666) in the output. Of course for variable size of truncation you need truncate() syscall. Let me know if you want this as an actual answer instead of the comment.– Sergiy Kolodyazhnyy
Jan 23 at 8:19
Just FYI,
open() and openat() already have a flag for truncating, O_TRUNC so technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that is command > file.txt, where file will be truncated if it exists. If you run strace on bash -c 'true > /dev/null' you'll see openat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666) in the output. Of course for variable size of truncation you need truncate() syscall. Let me know if you want this as an actual answer instead of the comment.– Sergiy Kolodyazhnyy
Jan 23 at 8:19
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18
add a comment |
2 Answers
2
active
oldest
votes
man -s 2 ftruncate says
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file
named by path or referenced by fd to be truncated to a size of precisely
length bytes.
...
CONFORMING TO
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
it goes on to say that if you use ftruncate you must have opened the file for writing, and if you use truncate the file must be writable.
edited Jan 23 at 10:57
ilkkachu
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
Appending to a file causes the OS to firsttruncateit to accommodate for the new file size, and thenwriteit. Sotruncateis the system call you're looking for.
– mgarciaisaia
Jan 23 at 12:42
1
On Linux, see alsofallocate(FALLOC_FL_COLLAPSE_RANGE)
– Stéphane Chazelas
Jan 23 at 17:40
add a comment |
The open(2) system call accepts the O_TRUNC flag that can reduce file size:
O_TRUNC– If the file exists and is a regular file, and the file is successfully openedO_RDWRorO_WRONLY, its length shall be truncated to 0, and the mode and owner shall be unchanged. It shall have no effect on FIFO special files or terminal device files. Its effect on other file types is implementation-defined. The result of usingO_TRUNCwithout eitherO_RDWRorO_WRONLYis undefined.
It is frequently used when the program aims to overwrite the content of a file entirely. An example is your shell’s file redirection operator as in command > file.
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f496104%2fdo-typical-system-call-interfaces-allow-reducing-the-size-of-a-file-without-rep%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
man -s 2 ftruncate says
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file
named by path or referenced by fd to be truncated to a size of precisely
length bytes.
...
CONFORMING TO
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
it goes on to say that if you use ftruncate you must have opened the file for writing, and if you use truncate the file must be writable.
edited Jan 23 at 10:57
ilkkachu
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
Appending to a file causes the OS to firsttruncateit to accommodate for the new file size, and thenwriteit. Sotruncateis the system call you're looking for.
– mgarciaisaia
Jan 23 at 12:42
1
On Linux, see alsofallocate(FALLOC_FL_COLLAPSE_RANGE)
– Stéphane Chazelas
Jan 23 at 17:40
add a comment |
man -s 2 ftruncate says
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file
named by path or referenced by fd to be truncated to a size of precisely
length bytes.
...
CONFORMING TO
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
it goes on to say that if you use ftruncate you must have opened the file for writing, and if you use truncate the file must be writable.
edited Jan 23 at 10:57
ilkkachu
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
Appending to a file causes the OS to firsttruncateit to accommodate for the new file size, and thenwriteit. Sotruncateis the system call you're looking for.
– mgarciaisaia
Jan 23 at 12:42
1
On Linux, see alsofallocate(FALLOC_FL_COLLAPSE_RANGE)
– Stéphane Chazelas
Jan 23 at 17:40
add a comment |
man -s 2 ftruncate says
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file
named by path or referenced by fd to be truncated to a size of precisely
length bytes.
...
CONFORMING TO
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
it goes on to say that if you use ftruncate you must have opened the file for writing, and if you use truncate the file must be writable.
edited Jan 23 at 10:57
ilkkachu
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
man -s 2 ftruncate says
DESCRIPTION
The truncate() and ftruncate() functions cause the regular file
named by path or referenced by fd to be truncated to a size of precisely
length bytes.
...
CONFORMING TO
POSIX.1-2001, POSIX.1-2008, 4.4BSD, SVr4 (these calls first appeared in 4.2BSD).
it goes on to say that if you use ftruncate you must have opened the file for writing, and if you use truncate the file must be writable.
edited Jan 23 at 10:57
ilkkachu
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
edited Jan 23 at 10:57
ilkkachu
61k1099174
edited Jan 23 at 10:57
ilkkachu
61k1099174
edited Jan 23 at 10:57
ilkkachu
61k1099174
61k1099174
answered Jan 23 at 2:14
icarusicarus
6,10111131
answered Jan 23 at 2:14
icarusicarus
6,10111131
answered Jan 23 at 2:14
icarusicarus
6,10111131
6,10111131
Appending to a file causes the OS to firsttruncateit to accommodate for the new file size, and thenwriteit. Sotruncateis the system call you're looking for.
– mgarciaisaia
Jan 23 at 12:42
1
On Linux, see alsofallocate(FALLOC_FL_COLLAPSE_RANGE)
– Stéphane Chazelas
Jan 23 at 17:40
add a comment |
Appending to a file causes the OS to firsttruncateit to accommodate for the new file size, and thenwriteit. Sotruncateis the system call you're looking for.
– mgarciaisaia
Jan 23 at 12:42
1
On Linux, see alsofallocate(FALLOC_FL_COLLAPSE_RANGE)
– Stéphane Chazelas
Jan 23 at 17:40
Appending to a file causes the OS to first
truncate it to accommodate for the new file size, and then write it. So truncate is the system call you're looking for.– mgarciaisaia
Jan 23 at 12:42
Appending to a file causes the OS to first
truncate it to accommodate for the new file size, and then write it. So truncate is the system call you're looking for.– mgarciaisaia
Jan 23 at 12:42
1
1
On Linux, see also
fallocate(FALLOC_FL_COLLAPSE_RANGE)– Stéphane Chazelas
Jan 23 at 17:40
On Linux, see also
fallocate(FALLOC_FL_COLLAPSE_RANGE)– Stéphane Chazelas
Jan 23 at 17:40
add a comment |
The open(2) system call accepts the O_TRUNC flag that can reduce file size:
O_TRUNC– If the file exists and is a regular file, and the file is successfully openedO_RDWRorO_WRONLY, its length shall be truncated to 0, and the mode and owner shall be unchanged. It shall have no effect on FIFO special files or terminal device files. Its effect on other file types is implementation-defined. The result of usingO_TRUNCwithout eitherO_RDWRorO_WRONLYis undefined.
It is frequently used when the program aims to overwrite the content of a file entirely. An example is your shell’s file redirection operator as in command > file.
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
add a comment |
The open(2) system call accepts the O_TRUNC flag that can reduce file size:
O_TRUNC– If the file exists and is a regular file, and the file is successfully openedO_RDWRorO_WRONLY, its length shall be truncated to 0, and the mode and owner shall be unchanged. It shall have no effect on FIFO special files or terminal device files. Its effect on other file types is implementation-defined. The result of usingO_TRUNCwithout eitherO_RDWRorO_WRONLYis undefined.
It is frequently used when the program aims to overwrite the content of a file entirely. An example is your shell’s file redirection operator as in command > file.
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
add a comment |
The open(2) system call accepts the O_TRUNC flag that can reduce file size:
O_TRUNC– If the file exists and is a regular file, and the file is successfully openedO_RDWRorO_WRONLY, its length shall be truncated to 0, and the mode and owner shall be unchanged. It shall have no effect on FIFO special files or terminal device files. Its effect on other file types is implementation-defined. The result of usingO_TRUNCwithout eitherO_RDWRorO_WRONLYis undefined.
It is frequently used when the program aims to overwrite the content of a file entirely. An example is your shell’s file redirection operator as in command > file.
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
The open(2) system call accepts the O_TRUNC flag that can reduce file size:
O_TRUNC– If the file exists and is a regular file, and the file is successfully openedO_RDWRorO_WRONLY, its length shall be truncated to 0, and the mode and owner shall be unchanged. It shall have no effect on FIFO special files or terminal device files. Its effect on other file types is implementation-defined. The result of usingO_TRUNCwithout eitherO_RDWRorO_WRONLYis undefined.
It is frequently used when the program aims to overwrite the content of a file entirely. An example is your shell’s file redirection operator as in command > file.
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
edited Jan 24 at 15:04
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
answered Jan 23 at 11:03
David FoersterDavid Foerster
1,019716
1,019716
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f496104%2fdo-typical-system-call-interfaces-allow-reducing-the-size-of-a-file-without-rep%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Just opening a file with
fopenin mode "w" (or "w+") will truncate it to zero length automatically. Or do you mean shrinking to a non-zero size, to preserve some of the old contents?– Wyzard
Jan 23 at 7:43
4
Just FYI,
open()andopenat()already have a flag for truncating,O_TRUNCso technically it does cause the file to shrink - that is, shrink completely - without changing inode. Most famous example of that iscommand > file.txt, where file will be truncated if it exists. If you runstraceonbash -c 'true > /dev/null'you'll seeopenat(AT_FDCWD, "/dev/null", O_WRONLY|O_CREAT|O_TRUNC, 0666)in the output. Of course for variable size of truncation you needtruncate()syscall. Let me know if you want this as an actual answer instead of the comment.– Sergiy Kolodyazhnyy
Jan 23 at 8:19
@SergiyKolodyazhnyy I was thinking of shrinking to an arbitrary size. I've upvoted David Foerster's answer because it answers a subset of the question, but I'm not accepting it in light of icarus's. I would have taken it as a yes-and-no type answer, whereas icarus's answer already showed that the real answer is a "yes".
– JoL
Jan 23 at 16:18