Mixing
find and replace closest values in a numpy array with respect to second array
up vote
1
down vote
favorite
I have a large 2D np.array (vec).
I would like to replace each value in vec with the closest value from a shorter array vals.
I have tried the following
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=0)]
but it does not work because the size of vec and vals are different.
Example input
vec = np.array([10.1,10.7,11.4,102,1100]
vals = np.array([10.0,11.0,100.0])
Desired output:
replaced_vals = [10.0,11.0,11.0,100.0,100.0]
python numpy
edited 45 mins ago
jpp
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
add a comment |
up vote
1
down vote
favorite
I have a large 2D np.array (vec).
I would like to replace each value in vec with the closest value from a shorter array vals.
I have tried the following
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=0)]
but it does not work because the size of vec and vals are different.
Example input
vec = np.array([10.1,10.7,11.4,102,1100]
vals = np.array([10.0,11.0,100.0])
Desired output:
replaced_vals = [10.0,11.0,11.0,100.0,100.0]
python numpy
edited 45 mins ago
jpp
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
Memory efficient solution-vals[closest_argmin(vec,vals)].
– Divakar
35 mins ago
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have a large 2D np.array (vec).
I would like to replace each value in vec with the closest value from a shorter array vals.
I have tried the following
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=0)]
but it does not work because the size of vec and vals are different.
Example input
vec = np.array([10.1,10.7,11.4,102,1100]
vals = np.array([10.0,11.0,100.0])
Desired output:
replaced_vals = [10.0,11.0,11.0,100.0,100.0]
python numpy
edited 45 mins ago
jpp
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
I have a large 2D np.array (vec).
I would like to replace each value in vec with the closest value from a shorter array vals.
I have tried the following
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=0)]
but it does not work because the size of vec and vals are different.
Example input
vec = np.array([10.1,10.7,11.4,102,1100]
vals = np.array([10.0,11.0,100.0])
Desired output:
replaced_vals = [10.0,11.0,11.0,100.0,100.0]
python numpy
python numpy
edited 45 mins ago
jpp
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
edited 45 mins ago
jpp
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
edited 45 mins ago
jpp
81.8k194795
edited 45 mins ago
jpp
81.8k194795
edited 45 mins ago
jpp
81.8k194795
81.8k194795
asked 47 mins ago
00__00__00
1,36811025
asked 47 mins ago
00__00__00
1,36811025
asked 47 mins ago
00__00__00
1,36811025
1,36811025
Memory efficient solution-vals[closest_argmin(vec,vals)].
– Divakar
35 mins ago
add a comment |
Memory efficient solution-vals[closest_argmin(vec,vals)].
– Divakar
35 mins ago
Memory efficient solution - vals[closest_argmin(vec,vals)].– Divakar
35 mins ago
Memory efficient solution - vals[closest_argmin(vec,vals)].– Divakar
35 mins ago
add a comment |
2 Answers
2
active
oldest
votes
up vote
2
down vote
You have to look along the other axis to get the desired values like this:
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
Output for your problem:
array([ 10., 11., 11., 100., 100.])
answered 44 mins ago
Thomas Pinetz
4,00411327
add a comment |
up vote
2
down vote
If your vals array is sorted, a more memory efficient, and possibly generally more efficient, solution is possible via np.searchsorted:
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
vec = np.array([10.1,10.7,11.4,102,1100])
vals = np.array([10.0,11.0,100.0])
print(jpp(vec, vals))
[ 10. 11. 11. 100. 100.]
Performance benchmarking
# Python 3.6.0, NumPy 1.11.3
n = 10**6
vec = np.array([10.1,10.7,11.4,102,1100]*n)
vals = np.array([10.0,11.0,100.0])
# @ThomasPinetz's solution, memory inefficient
def tho(vec, vals):
return vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
# @Divakar's solution, adapted from first related Q&A link
def diva(A, B):
L = B.size
sorted_idx = np.searchsorted(B, A)
sorted_idx[sorted_idx==L] = L-1
mask = (sorted_idx > 0) &
((np.abs(A - B[sorted_idx-1]) < np.abs(A - B[sorted_idx])) )
return B[sorted_idx-mask]
assert np.array_equal(tho(vec, vals), jpp(vec, vals))
assert np.array_equal(tho(vec, vals), diva(vec, vals))
%timeit tho(vec, vals) # 366 ms per loop
%timeit jpp(vec, vals) # 295 ms per loop
%timeit diva(vec, vals) # 334 ms per loop
Related Q&A
- Find nearest indices for one array against all values in another array - Python / NumPy
- Find nearest value in numpy array
answered 25 mins ago
jpp
81.8k194795
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
You have to look along the other axis to get the desired values like this:
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
Output for your problem:
array([ 10., 11., 11., 100., 100.])
answered 44 mins ago
Thomas Pinetz
4,00411327
add a comment |
up vote
2
down vote
You have to look along the other axis to get the desired values like this:
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
Output for your problem:
array([ 10., 11., 11., 100., 100.])
answered 44 mins ago
Thomas Pinetz
4,00411327
add a comment |
up vote
2
down vote
up vote
2
down vote
You have to look along the other axis to get the desired values like this:
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
Output for your problem:
array([ 10., 11., 11., 100., 100.])
answered 44 mins ago
Thomas Pinetz
4,00411327
You have to look along the other axis to get the desired values like this:
replaced_vals=vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
Output for your problem:
array([ 10., 11., 11., 100., 100.])
answered 44 mins ago
Thomas Pinetz
4,00411327
answered 44 mins ago
Thomas Pinetz
4,00411327
answered 44 mins ago
Thomas Pinetz
4,00411327
answered 44 mins ago
Thomas Pinetz
4,00411327
4,00411327
add a comment |
add a comment |
up vote
2
down vote
If your vals array is sorted, a more memory efficient, and possibly generally more efficient, solution is possible via np.searchsorted:
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
vec = np.array([10.1,10.7,11.4,102,1100])
vals = np.array([10.0,11.0,100.0])
print(jpp(vec, vals))
[ 10. 11. 11. 100. 100.]
Performance benchmarking
# Python 3.6.0, NumPy 1.11.3
n = 10**6
vec = np.array([10.1,10.7,11.4,102,1100]*n)
vals = np.array([10.0,11.0,100.0])
# @ThomasPinetz's solution, memory inefficient
def tho(vec, vals):
return vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
# @Divakar's solution, adapted from first related Q&A link
def diva(A, B):
L = B.size
sorted_idx = np.searchsorted(B, A)
sorted_idx[sorted_idx==L] = L-1
mask = (sorted_idx > 0) &
((np.abs(A - B[sorted_idx-1]) < np.abs(A - B[sorted_idx])) )
return B[sorted_idx-mask]
assert np.array_equal(tho(vec, vals), jpp(vec, vals))
assert np.array_equal(tho(vec, vals), diva(vec, vals))
%timeit tho(vec, vals) # 366 ms per loop
%timeit jpp(vec, vals) # 295 ms per loop
%timeit diva(vec, vals) # 334 ms per loop
Related Q&A
- Find nearest indices for one array against all values in another array - Python / NumPy
- Find nearest value in numpy array
answered 25 mins ago
jpp
81.8k194795
add a comment |
up vote
2
down vote
If your vals array is sorted, a more memory efficient, and possibly generally more efficient, solution is possible via np.searchsorted:
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
vec = np.array([10.1,10.7,11.4,102,1100])
vals = np.array([10.0,11.0,100.0])
print(jpp(vec, vals))
[ 10. 11. 11. 100. 100.]
Performance benchmarking
# Python 3.6.0, NumPy 1.11.3
n = 10**6
vec = np.array([10.1,10.7,11.4,102,1100]*n)
vals = np.array([10.0,11.0,100.0])
# @ThomasPinetz's solution, memory inefficient
def tho(vec, vals):
return vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
# @Divakar's solution, adapted from first related Q&A link
def diva(A, B):
L = B.size
sorted_idx = np.searchsorted(B, A)
sorted_idx[sorted_idx==L] = L-1
mask = (sorted_idx > 0) &
((np.abs(A - B[sorted_idx-1]) < np.abs(A - B[sorted_idx])) )
return B[sorted_idx-mask]
assert np.array_equal(tho(vec, vals), jpp(vec, vals))
assert np.array_equal(tho(vec, vals), diva(vec, vals))
%timeit tho(vec, vals) # 366 ms per loop
%timeit jpp(vec, vals) # 295 ms per loop
%timeit diva(vec, vals) # 334 ms per loop
Related Q&A
- Find nearest indices for one array against all values in another array - Python / NumPy
- Find nearest value in numpy array
answered 25 mins ago
jpp
81.8k194795
add a comment |
up vote
2
down vote
up vote
2
down vote
If your vals array is sorted, a more memory efficient, and possibly generally more efficient, solution is possible via np.searchsorted:
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
vec = np.array([10.1,10.7,11.4,102,1100])
vals = np.array([10.0,11.0,100.0])
print(jpp(vec, vals))
[ 10. 11. 11. 100. 100.]
Performance benchmarking
# Python 3.6.0, NumPy 1.11.3
n = 10**6
vec = np.array([10.1,10.7,11.4,102,1100]*n)
vals = np.array([10.0,11.0,100.0])
# @ThomasPinetz's solution, memory inefficient
def tho(vec, vals):
return vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
# @Divakar's solution, adapted from first related Q&A link
def diva(A, B):
L = B.size
sorted_idx = np.searchsorted(B, A)
sorted_idx[sorted_idx==L] = L-1
mask = (sorted_idx > 0) &
((np.abs(A - B[sorted_idx-1]) < np.abs(A - B[sorted_idx])) )
return B[sorted_idx-mask]
assert np.array_equal(tho(vec, vals), jpp(vec, vals))
assert np.array_equal(tho(vec, vals), diva(vec, vals))
%timeit tho(vec, vals) # 366 ms per loop
%timeit jpp(vec, vals) # 295 ms per loop
%timeit diva(vec, vals) # 334 ms per loop
Related Q&A
- Find nearest indices for one array against all values in another array - Python / NumPy
- Find nearest value in numpy array
answered 25 mins ago
jpp
81.8k194795
If your vals array is sorted, a more memory efficient, and possibly generally more efficient, solution is possible via np.searchsorted:
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
vec = np.array([10.1,10.7,11.4,102,1100])
vals = np.array([10.0,11.0,100.0])
print(jpp(vec, vals))
[ 10. 11. 11. 100. 100.]
Performance benchmarking
# Python 3.6.0, NumPy 1.11.3
n = 10**6
vec = np.array([10.1,10.7,11.4,102,1100]*n)
vals = np.array([10.0,11.0,100.0])
# @ThomasPinetz's solution, memory inefficient
def tho(vec, vals):
return vals[np.argmin(np.abs(vec[:, np.newaxis] - vals), axis=1)]
def jpp(vec, vals):
ss = np.searchsorted(vals, vec)
a = vals[ss - 1]
b = vals[np.minimum(len(vals) - 1, ss)]
return np.where(np.fabs(vec - a) < np.fabs(vec - b), a, b)
# @Divakar's solution, adapted from first related Q&A link
def diva(A, B):
L = B.size
sorted_idx = np.searchsorted(B, A)
sorted_idx[sorted_idx==L] = L-1
mask = (sorted_idx > 0) &
((np.abs(A - B[sorted_idx-1]) < np.abs(A - B[sorted_idx])) )
return B[sorted_idx-mask]
assert np.array_equal(tho(vec, vals), jpp(vec, vals))
assert np.array_equal(tho(vec, vals), diva(vec, vals))
%timeit tho(vec, vals) # 366 ms per loop
%timeit jpp(vec, vals) # 295 ms per loop
%timeit diva(vec, vals) # 334 ms per loop
Related Q&A
- Find nearest indices for one array against all values in another array - Python / NumPy
- Find nearest value in numpy array
answered 25 mins ago
jpp
81.8k194795
edited 2 mins ago
answered 25 mins ago
jpp
81.8k194795
answered 25 mins ago
jpp
81.8k194795
answered 25 mins ago
jpp
81.8k194795
81.8k194795
add a comment |
add a comment |
draft saved
draft discarded
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53371580%2ffind-and-replace-closest-values-in-a-numpy-array-with-respect-to-second-array%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Memory efficient solution-vals[closest_argmin(vec,vals)].– Divakar
35 mins ago