Are compilers allowed to optimize-out realloc?
up vote
26
down vote
favorite
I came across a situation where it would be useful to have unnecessary calls to realloc
being optimized out. However, it seems like neither clang nor gcc do such a thing (godbolt). - Although I see optimizations being made with multiple calls to malloc
.
The example:
void *myfunc() {
void *data;
data = malloc(100);
data = realloc(data, 200);
return data;
}
I expected it to be optimized to something like the following:
void *myfunc() {
return malloc(200);
}
Why is neither clang nor gcc optimizing it out? - Are they not allowed to do so?
c gcc clang language-lawyer compiler-optimization
New contributor
|
show 8 more comments
up vote
26
down vote
favorite
I came across a situation where it would be useful to have unnecessary calls to realloc
being optimized out. However, it seems like neither clang nor gcc do such a thing (godbolt). - Although I see optimizations being made with multiple calls to malloc
.
The example:
void *myfunc() {
void *data;
data = malloc(100);
data = realloc(data, 200);
return data;
}
I expected it to be optimized to something like the following:
void *myfunc() {
return malloc(200);
}
Why is neither clang nor gcc optimizing it out? - Are they not allowed to do so?
c gcc clang language-lawyer compiler-optimization
New contributor
10
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implementsmalloc
?
– Gerhardh
yesterday
8
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
8
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
3
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
11
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago
|
show 8 more comments
up vote
26
down vote
favorite
up vote
26
down vote
favorite
I came across a situation where it would be useful to have unnecessary calls to realloc
being optimized out. However, it seems like neither clang nor gcc do such a thing (godbolt). - Although I see optimizations being made with multiple calls to malloc
.
The example:
void *myfunc() {
void *data;
data = malloc(100);
data = realloc(data, 200);
return data;
}
I expected it to be optimized to something like the following:
void *myfunc() {
return malloc(200);
}
Why is neither clang nor gcc optimizing it out? - Are they not allowed to do so?
c gcc clang language-lawyer compiler-optimization
New contributor
I came across a situation where it would be useful to have unnecessary calls to realloc
being optimized out. However, it seems like neither clang nor gcc do such a thing (godbolt). - Although I see optimizations being made with multiple calls to malloc
.
The example:
void *myfunc() {
void *data;
data = malloc(100);
data = realloc(data, 200);
return data;
}
I expected it to be optimized to something like the following:
void *myfunc() {
return malloc(200);
}
Why is neither clang nor gcc optimizing it out? - Are they not allowed to do so?
c gcc clang language-lawyer compiler-optimization
c gcc clang language-lawyer compiler-optimization
New contributor
New contributor
edited yesterday
Florian Weimer
14.1k2943
14.1k2943
New contributor
asked yesterday
Julius
13124
13124
New contributor
New contributor
10
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implementsmalloc
?
– Gerhardh
yesterday
8
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
8
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
3
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
11
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago
|
show 8 more comments
10
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implementsmalloc
?
– Gerhardh
yesterday
8
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
8
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
3
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
11
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago
10
10
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implements
malloc
?– Gerhardh
yesterday
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implements
malloc
?– Gerhardh
yesterday
8
8
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
8
8
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
3
3
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
11
11
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago
|
show 8 more comments
5 Answers
5
active
oldest
votes
up vote
16
down vote
Are they not allowed to do so?
Maybe, but optimization not done in this case may be due to corner functional differences.
If 150 bytes of allocatable memory remain,data = malloc(100); data = realloc(data, 200);
returns NULL
with 100 bytes consumed (and leaked) and 50 remain.
data = malloc(200);
returns NULL
with 0 bytes consumed (none leaked) and 150 remain.
Different functionality in this narrow case may prevent optimization.
Are compilers allowed to optimize-out realloc?
Perhaps - I would expect it is allowed. Yet it may not be worth the effect to enhance the compiler to determine when it can.
Successful malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory.
It might be beyond that compiler's design to insure ...
, even if empty code, did not set any memory.
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started withchar *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. Withrealloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.
– chux
19 hours ago
1
@Julius IOWs, successfulmalloc(n); ... realloc(p, 2*n)
differs frommalloc(2*n);
when...
may have set some of the memory. It might be beyond that compiler's design to insure...
code did not set any memory.
– chux
19 hours ago
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
How doesmalloc(n); ... realloc(p, 2*n)
differ frommalloc(2*n); ...
?
– Andrew Svietlichnyy
16 hours ago
|
show 10 more comments
up vote
6
down vote
A compiler which bundles its own self-contained versions of malloc/calloc/free/realloc could legitimately perform the indicated optimization if the authors thought doing so was worth the effort. A compiler that chains to externally-supplied functions could still perform such optimizations if it documented that it did not regard the precise sequence of calls to such functions as an observable side-effect, but such treatment could be a bit more tenuous.
If no storage is allocated or deallocated between the malloc() and realloc(), the size of the realloc() is known when the malloc() is performed, and the realloc() size is larger than the malloc() size, then it may make sense to consolidate the malloc() and realloc() operations into a single larger allocation. If the state of memory could change in the interim, however, then such an optimization might cause the failure of operations that should have succeeded. For example, given the sequence:
void *p1 = malloc(2000000000);
void *p2 = malloc(2);
free(p1);
p2 = realloc(p2, 2000000000);
a system might not have 2000000000 bytes available for p2 until after p1 is freed. If it were to change the code to:
void *p1 = malloc(2000000000);
void *p2 = malloc(2000000000);
free(p1);
that would result in the allocation of p2 failing. Because the Standard never guarantees that allocation requests will succeed, such behavior would not be non-conforming. On the other hand, the following would also be a "conforming" implementation:
void *malloc(size_t size) { return 0; }
void *calloc(size_t size, size_t count) { return 0; }
void free(void *p) { }
void *realloc(void *p, size_t size) { return 0; }
Such an implementation might arguably be regarded as more "efficient" than most others, but one would have to be rather obtuse to regard it as being very useful except, perhaps, in rare situations where the above functions are are called on code paths that are never executed.
I think the Standard would clearly allow the optimization, at least in cases that are as simple as those in the original question. Even in cases where it might cause operations to fail that could otherwise have succeeded, the Standard would still allow it. Most likely, the reason that many compilers don't perform the optimization is that the authors didn't think the benefits would be sufficient to justify the effort required to identify cases where it would be safe and useful.
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whetherrealloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?
– supercat
15 hours ago
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed torealloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.
– Greg A. Woods
14 hours ago
add a comment |
up vote
3
down vote
The compiler is allowed to optimize out multiple calls to functions which are considered pure functions, i.e. functions that do not have any side-effects.
So the question is whether realloc()
is a pure function or not.
The C11 Standard Committee Draft N1570 states this about the realloc
function:
7.22.3.5 The realloc function
...
2. Therealloc
function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
Returns
4. Therealloc
function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Notice that the compiler cannot predict at compile time the value of the pointer that will be returned on each call.
This means that realloc()
cannot be considered a pure function and multiple calls to it cannot be optimized out by the compiler.
4
Freeing a block of memory has no observable side effects (a conforming compiler can implementfree
as a no-op and a conforming program won't be able to tell).
– n.m.
yesterday
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
@AndrewHenle: The behavior of callingfree
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.
– Eric Postpischil
yesterday
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior ofmalloc
andrealloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.
– Eric Postpischil
yesterday
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
|
show 26 more comments
up vote
0
down vote
Yes, I think this is allowed by N3664
That appears to be primarily related to C++ (particularly, thenew
operator).
– usr
yesterday
1
@usr clang claims to have implemented N3664, however, since C++ doesn't haverealloc
(in terms ofnew
/delete
) it might explain whymalloc
/free
calls are optimized andrealloc
isn't?
– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
@usr That's just the legacy-wrapper for therealloc()
C function, and is not interoperable withnew/delete
. Even when combined withstd::malloc()
andstd::free()
,std::realloc()
won't call any constructors/destructors the waystd::vector<>
does, so it's pretty much worthless for modern C++ programmers...
– cmaster
19 hours ago
|
show 2 more comments
up vote
0
down vote
But you're not checking the return value of the first malloc() which you're then using in the second realloc(). It could just as well be NULL.
How could the compiler optimize the two calls into a single one without making unwarranted assumptions about the return value of the first?
Then there is another possible scenario. FreeBSD used to have a realloc()
which was basically malloc + memcpy + free the old pointer.
Suppose that there are only 230 bytes left of free memory. In that implementation, ptr = malloc(100)
followed by realloc(ptr, 200)
will fail, but a single malloc(200)
will succeed.
New contributor
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
add a comment |
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
16
down vote
Are they not allowed to do so?
Maybe, but optimization not done in this case may be due to corner functional differences.
If 150 bytes of allocatable memory remain,data = malloc(100); data = realloc(data, 200);
returns NULL
with 100 bytes consumed (and leaked) and 50 remain.
data = malloc(200);
returns NULL
with 0 bytes consumed (none leaked) and 150 remain.
Different functionality in this narrow case may prevent optimization.
Are compilers allowed to optimize-out realloc?
Perhaps - I would expect it is allowed. Yet it may not be worth the effect to enhance the compiler to determine when it can.
Successful malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory.
It might be beyond that compiler's design to insure ...
, even if empty code, did not set any memory.
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started withchar *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. Withrealloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.
– chux
19 hours ago
1
@Julius IOWs, successfulmalloc(n); ... realloc(p, 2*n)
differs frommalloc(2*n);
when...
may have set some of the memory. It might be beyond that compiler's design to insure...
code did not set any memory.
– chux
19 hours ago
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
How doesmalloc(n); ... realloc(p, 2*n)
differ frommalloc(2*n); ...
?
– Andrew Svietlichnyy
16 hours ago
|
show 10 more comments
up vote
16
down vote
Are they not allowed to do so?
Maybe, but optimization not done in this case may be due to corner functional differences.
If 150 bytes of allocatable memory remain,data = malloc(100); data = realloc(data, 200);
returns NULL
with 100 bytes consumed (and leaked) and 50 remain.
data = malloc(200);
returns NULL
with 0 bytes consumed (none leaked) and 150 remain.
Different functionality in this narrow case may prevent optimization.
Are compilers allowed to optimize-out realloc?
Perhaps - I would expect it is allowed. Yet it may not be worth the effect to enhance the compiler to determine when it can.
Successful malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory.
It might be beyond that compiler's design to insure ...
, even if empty code, did not set any memory.
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started withchar *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. Withrealloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.
– chux
19 hours ago
1
@Julius IOWs, successfulmalloc(n); ... realloc(p, 2*n)
differs frommalloc(2*n);
when...
may have set some of the memory. It might be beyond that compiler's design to insure...
code did not set any memory.
– chux
19 hours ago
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
How doesmalloc(n); ... realloc(p, 2*n)
differ frommalloc(2*n); ...
?
– Andrew Svietlichnyy
16 hours ago
|
show 10 more comments
up vote
16
down vote
up vote
16
down vote
Are they not allowed to do so?
Maybe, but optimization not done in this case may be due to corner functional differences.
If 150 bytes of allocatable memory remain,data = malloc(100); data = realloc(data, 200);
returns NULL
with 100 bytes consumed (and leaked) and 50 remain.
data = malloc(200);
returns NULL
with 0 bytes consumed (none leaked) and 150 remain.
Different functionality in this narrow case may prevent optimization.
Are compilers allowed to optimize-out realloc?
Perhaps - I would expect it is allowed. Yet it may not be worth the effect to enhance the compiler to determine when it can.
Successful malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory.
It might be beyond that compiler's design to insure ...
, even if empty code, did not set any memory.
Are they not allowed to do so?
Maybe, but optimization not done in this case may be due to corner functional differences.
If 150 bytes of allocatable memory remain,data = malloc(100); data = realloc(data, 200);
returns NULL
with 100 bytes consumed (and leaked) and 50 remain.
data = malloc(200);
returns NULL
with 0 bytes consumed (none leaked) and 150 remain.
Different functionality in this narrow case may prevent optimization.
Are compilers allowed to optimize-out realloc?
Perhaps - I would expect it is allowed. Yet it may not be worth the effect to enhance the compiler to determine when it can.
Successful malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory.
It might be beyond that compiler's design to insure ...
, even if empty code, did not set any memory.
edited 17 hours ago
answered 23 hours ago
chux
78.3k869143
78.3k869143
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started withchar *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. Withrealloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.
– chux
19 hours ago
1
@Julius IOWs, successfulmalloc(n); ... realloc(p, 2*n)
differs frommalloc(2*n);
when...
may have set some of the memory. It might be beyond that compiler's design to insure...
code did not set any memory.
– chux
19 hours ago
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
How doesmalloc(n); ... realloc(p, 2*n)
differ frommalloc(2*n); ...
?
– Andrew Svietlichnyy
16 hours ago
|
show 10 more comments
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started withchar *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. Withrealloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.
– chux
19 hours ago
1
@Julius IOWs, successfulmalloc(n); ... realloc(p, 2*n)
differs frommalloc(2*n);
when...
may have set some of the memory. It might be beyond that compiler's design to insure...
code did not set any memory.
– chux
19 hours ago
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
How doesmalloc(n); ... realloc(p, 2*n)
differ frommalloc(2*n); ...
?
– Andrew Svietlichnyy
16 hours ago
2
2
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
I was thinking of that as well. However, this example shows that a realloc between prevents the malloc/free from being optimized out. If you remove it, the compiler will optimize-out the malloc and free. - and as far as I can see there will be no difference in the result?
– Julius
23 hours ago
2
2
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started with
char *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. With realloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.– chux
19 hours ago
@Julius Per this code , I see no reason disbarring optimization. Yet consider if both codes started with
char *data = malloc(100); if (data == NULL) { return NULL; } *data = 1
, the functions are different. With realloc()
copying the first 100 bytes, the compiler may not see that copying uninitialized was not important in your code. BTW: 2nd compiler is C++ not C. Suggest comparing C to C.– chux
19 hours ago
1
1
@Julius IOWs, successful
malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory. It might be beyond that compiler's design to insure ...
code did not set any memory.– chux
19 hours ago
@Julius IOWs, successful
malloc(n); ... realloc(p, 2*n)
differs from malloc(2*n);
when ...
may have set some of the memory. It might be beyond that compiler's design to insure ...
code did not set any memory.– chux
19 hours ago
2
2
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
@chux That's an interesting thought. I could imagine it to be quite difficult to prove that there were no changes to the specific memory region in some cases - although it is most likely simple in other cases.
– Julius
18 hours ago
2
2
How does
malloc(n); ... realloc(p, 2*n)
differ from malloc(2*n); ...
?– Andrew Svietlichnyy
16 hours ago
How does
malloc(n); ... realloc(p, 2*n)
differ from malloc(2*n); ...
?– Andrew Svietlichnyy
16 hours ago
|
show 10 more comments
up vote
6
down vote
A compiler which bundles its own self-contained versions of malloc/calloc/free/realloc could legitimately perform the indicated optimization if the authors thought doing so was worth the effort. A compiler that chains to externally-supplied functions could still perform such optimizations if it documented that it did not regard the precise sequence of calls to such functions as an observable side-effect, but such treatment could be a bit more tenuous.
If no storage is allocated or deallocated between the malloc() and realloc(), the size of the realloc() is known when the malloc() is performed, and the realloc() size is larger than the malloc() size, then it may make sense to consolidate the malloc() and realloc() operations into a single larger allocation. If the state of memory could change in the interim, however, then such an optimization might cause the failure of operations that should have succeeded. For example, given the sequence:
void *p1 = malloc(2000000000);
void *p2 = malloc(2);
free(p1);
p2 = realloc(p2, 2000000000);
a system might not have 2000000000 bytes available for p2 until after p1 is freed. If it were to change the code to:
void *p1 = malloc(2000000000);
void *p2 = malloc(2000000000);
free(p1);
that would result in the allocation of p2 failing. Because the Standard never guarantees that allocation requests will succeed, such behavior would not be non-conforming. On the other hand, the following would also be a "conforming" implementation:
void *malloc(size_t size) { return 0; }
void *calloc(size_t size, size_t count) { return 0; }
void free(void *p) { }
void *realloc(void *p, size_t size) { return 0; }
Such an implementation might arguably be regarded as more "efficient" than most others, but one would have to be rather obtuse to regard it as being very useful except, perhaps, in rare situations where the above functions are are called on code paths that are never executed.
I think the Standard would clearly allow the optimization, at least in cases that are as simple as those in the original question. Even in cases where it might cause operations to fail that could otherwise have succeeded, the Standard would still allow it. Most likely, the reason that many compilers don't perform the optimization is that the authors didn't think the benefits would be sufficient to justify the effort required to identify cases where it would be safe and useful.
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whetherrealloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?
– supercat
15 hours ago
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed torealloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.
– Greg A. Woods
14 hours ago
add a comment |
up vote
6
down vote
A compiler which bundles its own self-contained versions of malloc/calloc/free/realloc could legitimately perform the indicated optimization if the authors thought doing so was worth the effort. A compiler that chains to externally-supplied functions could still perform such optimizations if it documented that it did not regard the precise sequence of calls to such functions as an observable side-effect, but such treatment could be a bit more tenuous.
If no storage is allocated or deallocated between the malloc() and realloc(), the size of the realloc() is known when the malloc() is performed, and the realloc() size is larger than the malloc() size, then it may make sense to consolidate the malloc() and realloc() operations into a single larger allocation. If the state of memory could change in the interim, however, then such an optimization might cause the failure of operations that should have succeeded. For example, given the sequence:
void *p1 = malloc(2000000000);
void *p2 = malloc(2);
free(p1);
p2 = realloc(p2, 2000000000);
a system might not have 2000000000 bytes available for p2 until after p1 is freed. If it were to change the code to:
void *p1 = malloc(2000000000);
void *p2 = malloc(2000000000);
free(p1);
that would result in the allocation of p2 failing. Because the Standard never guarantees that allocation requests will succeed, such behavior would not be non-conforming. On the other hand, the following would also be a "conforming" implementation:
void *malloc(size_t size) { return 0; }
void *calloc(size_t size, size_t count) { return 0; }
void free(void *p) { }
void *realloc(void *p, size_t size) { return 0; }
Such an implementation might arguably be regarded as more "efficient" than most others, but one would have to be rather obtuse to regard it as being very useful except, perhaps, in rare situations where the above functions are are called on code paths that are never executed.
I think the Standard would clearly allow the optimization, at least in cases that are as simple as those in the original question. Even in cases where it might cause operations to fail that could otherwise have succeeded, the Standard would still allow it. Most likely, the reason that many compilers don't perform the optimization is that the authors didn't think the benefits would be sufficient to justify the effort required to identify cases where it would be safe and useful.
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whetherrealloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?
– supercat
15 hours ago
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed torealloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.
– Greg A. Woods
14 hours ago
add a comment |
up vote
6
down vote
up vote
6
down vote
A compiler which bundles its own self-contained versions of malloc/calloc/free/realloc could legitimately perform the indicated optimization if the authors thought doing so was worth the effort. A compiler that chains to externally-supplied functions could still perform such optimizations if it documented that it did not regard the precise sequence of calls to such functions as an observable side-effect, but such treatment could be a bit more tenuous.
If no storage is allocated or deallocated between the malloc() and realloc(), the size of the realloc() is known when the malloc() is performed, and the realloc() size is larger than the malloc() size, then it may make sense to consolidate the malloc() and realloc() operations into a single larger allocation. If the state of memory could change in the interim, however, then such an optimization might cause the failure of operations that should have succeeded. For example, given the sequence:
void *p1 = malloc(2000000000);
void *p2 = malloc(2);
free(p1);
p2 = realloc(p2, 2000000000);
a system might not have 2000000000 bytes available for p2 until after p1 is freed. If it were to change the code to:
void *p1 = malloc(2000000000);
void *p2 = malloc(2000000000);
free(p1);
that would result in the allocation of p2 failing. Because the Standard never guarantees that allocation requests will succeed, such behavior would not be non-conforming. On the other hand, the following would also be a "conforming" implementation:
void *malloc(size_t size) { return 0; }
void *calloc(size_t size, size_t count) { return 0; }
void free(void *p) { }
void *realloc(void *p, size_t size) { return 0; }
Such an implementation might arguably be regarded as more "efficient" than most others, but one would have to be rather obtuse to regard it as being very useful except, perhaps, in rare situations where the above functions are are called on code paths that are never executed.
I think the Standard would clearly allow the optimization, at least in cases that are as simple as those in the original question. Even in cases where it might cause operations to fail that could otherwise have succeeded, the Standard would still allow it. Most likely, the reason that many compilers don't perform the optimization is that the authors didn't think the benefits would be sufficient to justify the effort required to identify cases where it would be safe and useful.
A compiler which bundles its own self-contained versions of malloc/calloc/free/realloc could legitimately perform the indicated optimization if the authors thought doing so was worth the effort. A compiler that chains to externally-supplied functions could still perform such optimizations if it documented that it did not regard the precise sequence of calls to such functions as an observable side-effect, but such treatment could be a bit more tenuous.
If no storage is allocated or deallocated between the malloc() and realloc(), the size of the realloc() is known when the malloc() is performed, and the realloc() size is larger than the malloc() size, then it may make sense to consolidate the malloc() and realloc() operations into a single larger allocation. If the state of memory could change in the interim, however, then such an optimization might cause the failure of operations that should have succeeded. For example, given the sequence:
void *p1 = malloc(2000000000);
void *p2 = malloc(2);
free(p1);
p2 = realloc(p2, 2000000000);
a system might not have 2000000000 bytes available for p2 until after p1 is freed. If it were to change the code to:
void *p1 = malloc(2000000000);
void *p2 = malloc(2000000000);
free(p1);
that would result in the allocation of p2 failing. Because the Standard never guarantees that allocation requests will succeed, such behavior would not be non-conforming. On the other hand, the following would also be a "conforming" implementation:
void *malloc(size_t size) { return 0; }
void *calloc(size_t size, size_t count) { return 0; }
void free(void *p) { }
void *realloc(void *p, size_t size) { return 0; }
Such an implementation might arguably be regarded as more "efficient" than most others, but one would have to be rather obtuse to regard it as being very useful except, perhaps, in rare situations where the above functions are are called on code paths that are never executed.
I think the Standard would clearly allow the optimization, at least in cases that are as simple as those in the original question. Even in cases where it might cause operations to fail that could otherwise have succeeded, the Standard would still allow it. Most likely, the reason that many compilers don't perform the optimization is that the authors didn't think the benefits would be sufficient to justify the effort required to identify cases where it would be safe and useful.
answered 19 hours ago
supercat
55.6k2115146
55.6k2115146
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whetherrealloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?
– supercat
15 hours ago
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed torealloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.
– Greg A. Woods
14 hours ago
add a comment |
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whetherrealloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?
– supercat
15 hours ago
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed torealloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.
– Greg A. Woods
14 hours ago
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
Standards wise, no. C99 and C11 explicitly state the old object is deallocated and a new object is returned. Even with a private allocator the compiler cannot predict at compile time the pointer to a new allocation.
– Greg A. Woods
15 hours ago
1
1
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whether
realloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?– supercat
15 hours ago
@GregA.Woods: Under the as-if rule, a compiler would be allowed to consolidate the operations if the resulting behavior could have resulted from doing the operations separately. By what standard-defined means could a program observe whether
realloc
actually did anything other than yield a pointer to an allocation that might have already been as big as requested?– supercat
15 hours ago
1
1
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed to
realloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.– Greg A. Woods
14 hours ago
I suppose if the compiler's own allocator was known to always initially allocate more, say ten times as much, space as is required, and if the compiler could predict at compile time that the the allocated object's desired size was X, and also predict that the new size passed to
realloc()
was less than 10X then it could assume the object would not change location. However I don't know if the optimization would still be allowed by the current standard. Perhaps.– Greg A. Woods
14 hours ago
add a comment |
up vote
3
down vote
The compiler is allowed to optimize out multiple calls to functions which are considered pure functions, i.e. functions that do not have any side-effects.
So the question is whether realloc()
is a pure function or not.
The C11 Standard Committee Draft N1570 states this about the realloc
function:
7.22.3.5 The realloc function
...
2. Therealloc
function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
Returns
4. Therealloc
function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Notice that the compiler cannot predict at compile time the value of the pointer that will be returned on each call.
This means that realloc()
cannot be considered a pure function and multiple calls to it cannot be optimized out by the compiler.
4
Freeing a block of memory has no observable side effects (a conforming compiler can implementfree
as a no-op and a conforming program won't be able to tell).
– n.m.
yesterday
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
@AndrewHenle: The behavior of callingfree
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.
– Eric Postpischil
yesterday
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior ofmalloc
andrealloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.
– Eric Postpischil
yesterday
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
|
show 26 more comments
up vote
3
down vote
The compiler is allowed to optimize out multiple calls to functions which are considered pure functions, i.e. functions that do not have any side-effects.
So the question is whether realloc()
is a pure function or not.
The C11 Standard Committee Draft N1570 states this about the realloc
function:
7.22.3.5 The realloc function
...
2. Therealloc
function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
Returns
4. Therealloc
function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Notice that the compiler cannot predict at compile time the value of the pointer that will be returned on each call.
This means that realloc()
cannot be considered a pure function and multiple calls to it cannot be optimized out by the compiler.
4
Freeing a block of memory has no observable side effects (a conforming compiler can implementfree
as a no-op and a conforming program won't be able to tell).
– n.m.
yesterday
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
@AndrewHenle: The behavior of callingfree
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.
– Eric Postpischil
yesterday
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior ofmalloc
andrealloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.
– Eric Postpischil
yesterday
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
|
show 26 more comments
up vote
3
down vote
up vote
3
down vote
The compiler is allowed to optimize out multiple calls to functions which are considered pure functions, i.e. functions that do not have any side-effects.
So the question is whether realloc()
is a pure function or not.
The C11 Standard Committee Draft N1570 states this about the realloc
function:
7.22.3.5 The realloc function
...
2. Therealloc
function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
Returns
4. Therealloc
function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Notice that the compiler cannot predict at compile time the value of the pointer that will be returned on each call.
This means that realloc()
cannot be considered a pure function and multiple calls to it cannot be optimized out by the compiler.
The compiler is allowed to optimize out multiple calls to functions which are considered pure functions, i.e. functions that do not have any side-effects.
So the question is whether realloc()
is a pure function or not.
The C11 Standard Committee Draft N1570 states this about the realloc
function:
7.22.3.5 The realloc function
...
2. Therealloc
function deallocates the old object pointed to by ptr and returns a pointer to a new object that has the size specified by size. The contents of the new object shall be the same as that of the old object prior to deallocation, up to the lesser of the new and old sizes. Any bytes in the new object beyond the size of the old object have indeterminate values.
Returns
4. Therealloc
function returns a pointer to the new object (which may have the same value as a pointer to the old object), or a null pointer if the new object could not be allocated.
Notice that the compiler cannot predict at compile time the value of the pointer that will be returned on each call.
This means that realloc()
cannot be considered a pure function and multiple calls to it cannot be optimized out by the compiler.
edited 7 hours ago
answered yesterday
P.W
7,5902438
7,5902438
4
Freeing a block of memory has no observable side effects (a conforming compiler can implementfree
as a no-op and a conforming program won't be able to tell).
– n.m.
yesterday
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
@AndrewHenle: The behavior of callingfree
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.
– Eric Postpischil
yesterday
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior ofmalloc
andrealloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.
– Eric Postpischil
yesterday
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
|
show 26 more comments
4
Freeing a block of memory has no observable side effects (a conforming compiler can implementfree
as a no-op and a conforming program won't be able to tell).
– n.m.
yesterday
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
@AndrewHenle: The behavior of callingfree
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.
– Eric Postpischil
yesterday
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior ofmalloc
andrealloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.
– Eric Postpischil
yesterday
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
4
4
Freeing a block of memory has no observable side effects (a conforming compiler can implement
free
as a no-op and a conforming program won't be able to tell).– n.m.
yesterday
Freeing a block of memory has no observable side effects (a conforming compiler can implement
free
as a no-op and a conforming program won't be able to tell).– n.m.
yesterday
2
2
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
@AndrewHenle but why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
yesterday
9
9
@AndrewHenle: The behavior of calling
free
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.– Eric Postpischil
yesterday
@AndrewHenle: The behavior of calling
free
on a previously freed block is undefined. Therefore, whatever the observable behavior occurs upon doing so cannot be used to deduce whether or not a previously free was or was not actually implemented—any behavior could be exhibited for any reason.– Eric Postpischil
yesterday
13
13
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior of
malloc
and realloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.– Eric Postpischil
yesterday
The compiler is allowed to optimize more than calls to pure functions. It is allowed to optimize anything that does not change the defined observable behavior. The behavior of
malloc
and realloc
is specified by the C standard, so the compiler may optimize them in ways that result in the same observable behavior in accordance with their specifications.– Eric Postpischil
yesterday
3
3
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
@AndrewHenle: It is not observable in this code since the two pointers are never compared. The two values do not even exist at the same time.
– Eric Postpischil
yesterday
|
show 26 more comments
up vote
0
down vote
Yes, I think this is allowed by N3664
That appears to be primarily related to C++ (particularly, thenew
operator).
– usr
yesterday
1
@usr clang claims to have implemented N3664, however, since C++ doesn't haverealloc
(in terms ofnew
/delete
) it might explain whymalloc
/free
calls are optimized andrealloc
isn't?
– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
@usr That's just the legacy-wrapper for therealloc()
C function, and is not interoperable withnew/delete
. Even when combined withstd::malloc()
andstd::free()
,std::realloc()
won't call any constructors/destructors the waystd::vector<>
does, so it's pretty much worthless for modern C++ programmers...
– cmaster
19 hours ago
|
show 2 more comments
up vote
0
down vote
Yes, I think this is allowed by N3664
That appears to be primarily related to C++ (particularly, thenew
operator).
– usr
yesterday
1
@usr clang claims to have implemented N3664, however, since C++ doesn't haverealloc
(in terms ofnew
/delete
) it might explain whymalloc
/free
calls are optimized andrealloc
isn't?
– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
@usr That's just the legacy-wrapper for therealloc()
C function, and is not interoperable withnew/delete
. Even when combined withstd::malloc()
andstd::free()
,std::realloc()
won't call any constructors/destructors the waystd::vector<>
does, so it's pretty much worthless for modern C++ programmers...
– cmaster
19 hours ago
|
show 2 more comments
up vote
0
down vote
up vote
0
down vote
Yes, I think this is allowed by N3664
Yes, I think this is allowed by N3664
answered yesterday
bogdan tudose
436
436
That appears to be primarily related to C++ (particularly, thenew
operator).
– usr
yesterday
1
@usr clang claims to have implemented N3664, however, since C++ doesn't haverealloc
(in terms ofnew
/delete
) it might explain whymalloc
/free
calls are optimized andrealloc
isn't?
– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
@usr That's just the legacy-wrapper for therealloc()
C function, and is not interoperable withnew/delete
. Even when combined withstd::malloc()
andstd::free()
,std::realloc()
won't call any constructors/destructors the waystd::vector<>
does, so it's pretty much worthless for modern C++ programmers...
– cmaster
19 hours ago
|
show 2 more comments
That appears to be primarily related to C++ (particularly, thenew
operator).
– usr
yesterday
1
@usr clang claims to have implemented N3664, however, since C++ doesn't haverealloc
(in terms ofnew
/delete
) it might explain whymalloc
/free
calls are optimized andrealloc
isn't?
– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
@usr That's just the legacy-wrapper for therealloc()
C function, and is not interoperable withnew/delete
. Even when combined withstd::malloc()
andstd::free()
,std::realloc()
won't call any constructors/destructors the waystd::vector<>
does, so it's pretty much worthless for modern C++ programmers...
– cmaster
19 hours ago
That appears to be primarily related to C++ (particularly, the
new
operator).– usr
yesterday
That appears to be primarily related to C++ (particularly, the
new
operator).– usr
yesterday
1
1
@usr clang claims to have implemented N3664, however, since C++ doesn't have
realloc
(in terms of new
/delete
) it might explain why malloc
/free
calls are optimized and realloc
isn't?– Julius
yesterday
@usr clang claims to have implemented N3664, however, since C++ doesn't have
realloc
(in terms of new
/delete
) it might explain why malloc
/free
calls are optimized and realloc
isn't?– Julius
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
@Julius C++ does have realloc. If it was a proposed change to relevant part of C standard, it'd be relevant.
– usr
yesterday
3
3
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
@usr Ah yes, you are right! I was thinking about an equivalent C++ operator such as new/delete. Stroustroup suggests to do reallocations through a container though.
– Julius
yesterday
1
1
@usr That's just the legacy-wrapper for the
realloc()
C function, and is not interoperable with new/delete
. Even when combined with std::malloc()
and std::free()
, std::realloc()
won't call any constructors/destructors the way std::vector<>
does, so it's pretty much worthless for modern C++ programmers...– cmaster
19 hours ago
@usr That's just the legacy-wrapper for the
realloc()
C function, and is not interoperable with new/delete
. Even when combined with std::malloc()
and std::free()
, std::realloc()
won't call any constructors/destructors the way std::vector<>
does, so it's pretty much worthless for modern C++ programmers...– cmaster
19 hours ago
|
show 2 more comments
up vote
0
down vote
But you're not checking the return value of the first malloc() which you're then using in the second realloc(). It could just as well be NULL.
How could the compiler optimize the two calls into a single one without making unwarranted assumptions about the return value of the first?
Then there is another possible scenario. FreeBSD used to have a realloc()
which was basically malloc + memcpy + free the old pointer.
Suppose that there are only 230 bytes left of free memory. In that implementation, ptr = malloc(100)
followed by realloc(ptr, 200)
will fail, but a single malloc(200)
will succeed.
New contributor
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
add a comment |
up vote
0
down vote
But you're not checking the return value of the first malloc() which you're then using in the second realloc(). It could just as well be NULL.
How could the compiler optimize the two calls into a single one without making unwarranted assumptions about the return value of the first?
Then there is another possible scenario. FreeBSD used to have a realloc()
which was basically malloc + memcpy + free the old pointer.
Suppose that there are only 230 bytes left of free memory. In that implementation, ptr = malloc(100)
followed by realloc(ptr, 200)
will fail, but a single malloc(200)
will succeed.
New contributor
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
add a comment |
up vote
0
down vote
up vote
0
down vote
But you're not checking the return value of the first malloc() which you're then using in the second realloc(). It could just as well be NULL.
How could the compiler optimize the two calls into a single one without making unwarranted assumptions about the return value of the first?
Then there is another possible scenario. FreeBSD used to have a realloc()
which was basically malloc + memcpy + free the old pointer.
Suppose that there are only 230 bytes left of free memory. In that implementation, ptr = malloc(100)
followed by realloc(ptr, 200)
will fail, but a single malloc(200)
will succeed.
New contributor
But you're not checking the return value of the first malloc() which you're then using in the second realloc(). It could just as well be NULL.
How could the compiler optimize the two calls into a single one without making unwarranted assumptions about the return value of the first?
Then there is another possible scenario. FreeBSD used to have a realloc()
which was basically malloc + memcpy + free the old pointer.
Suppose that there are only 230 bytes left of free memory. In that implementation, ptr = malloc(100)
followed by realloc(ptr, 200)
will fail, but a single malloc(200)
will succeed.
New contributor
New contributor
answered 6 hours ago
pizdelect
1
1
New contributor
New contributor
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
add a comment |
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
You are right about the checking, but I have submitted at least one example in the comments which include checking the return value - it doesn't seem to make a difference. Actually, the compiler does make such assumptions sometimes which I could demonstrate.
– Julius
4 hours ago
add a comment |
Julius is a new contributor. Be nice, and check out our Code of Conduct.
Julius is a new contributor. Be nice, and check out our Code of Conduct.
Julius is a new contributor. Be nice, and check out our Code of Conduct.
Julius is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53373421%2fare-compilers-allowed-to-optimize-out-realloc%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
10
I would be really surprised it a compiler was allowed to remove calls to external functions. What if you link with your own library that implements
malloc
?– Gerhardh
yesterday
8
@Gerhardh malloc is not an external function, it's a part of the standard library. Compilers are allowed to inline it or otherwise implement it however they wish.
– n.m.
yesterday
8
@Lundin: It is not true that a compiler is not allowed to optimize out a function call if the function contains any side effects. A compiler is not allowed to optimize away observable behavior. If a side effect (and its consequences) is not observable, it may be removed.
– Eric Postpischil
yesterday
3
@Lundin It would make sense, but, why are two consecutive calls to malloc/free optimized out (godbolt.org/z/gBVXcp)? That wouldn't be allowed if it had a side effect, would it?
– Julius
23 hours ago
11
@Lundin: An unobservable side effect is not needed.
– Eric Postpischil
23 hours ago