python - Type casting error with numpy.take -

python - Type casting error with numpy.take -

i have look-up table (lut) stores 65536 uint8 values:

lut = np.random.randint(256, size=(65536,)).astype('uint8')

i want utilize lut convert values in array of uint16s:

arr = np.random.randint(65536, size=(1000, 1000)).astype('uint16')

and want conversion in place, because lastly array can pretty big. when seek it, next happens:

>>> np.take(lut, arr, out=arr) traceback (most recent phone call last): file "<stdin>", line 1, in <module> file "c:\python27\lib\site-packages\numpy\core\fromnumeric.py", line 103, in take homecoming take(indices, axis, out, mode) typeerror: array cannot safely cast required type

and don't understand going on. know that, without out argument, homecoming of same dtype lut, uint8. why can't uint8 cast uint16? if inquire numpy:

>>> np.can_cast('uint8', 'uint16') true

obviously next works:

>>> lut = lut.astype('uint16') >>> np.take(lut, arr, out=arr) array([[173, 251, 218, ..., 110, 98, 235], [200, 231, 91, ..., 158, 100, 88], [ 13, 227, 223, ..., 94, 56, 36], ..., [ 28, 198, 80, ..., 60, 87, 118], [156, 46, 118, ..., 212, 198, 218], [203, 97, 245, ..., 3, 191, 173]], dtype=uint16)

but works:

>>> lut = lut.astype('int32') >>> np.take(lut, arr, out=arr) array([[ 78, 249, 148, ..., 77, 12, 167], [138, 5, 206, ..., 31, 43, 244], [ 29, 134, 131, ..., 100, 107, 1], ..., [109, 166, 14, ..., 64, 95, 102], [152, 169, 102, ..., 240, 166, 148], [ 47, 14, 129, ..., 237, 11, 78]], dtype=uint16)

this makes no sense, since int32s beingness cast uint16s, not safe thing do:

>>> np.can_cast('int32', 'uint16') false

my code works if set lut's dtype in uint16, uint32, uint64, int32 or int64, fails uint8, int8 , int16.

am missing something, or broken in numpy?

workarounds welcome... since lut not big, guess not bad have type match array's, if takes twice space, doesn't sense right that...

is there way tell numpy not worry casting safety?

interesting problem. numpy.take(lut, ...) gets transformed lut.take(...) source can looked @ here:

https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/item_selection.c#l28

i believe exception thrown at line 105:

obj = (pyarrayobject *)pyarray_fromarray(out, dtype, flags); if (obj == null) { goto fail; }

where in case out arr dtype 1 of lut, i.e. uint8. tries cast arr uint8, fails. have i'm not sure why needs that, pointing out does... reason take seems assume want output array have same dtype lut.

by way, in many cases phone call pyarray_fromarray create new array , replacement not in place. case illustration if phone call take mode='raise' (the default, , happens in examples), or whenever lut.dtype != arr.dtype. well, @ to the lowest degree should, , can't explain why, when cast lut int32 output array remains uint16! mystery me - maybe has npy_array_updateifcopy flag (see here).

bottom line:

the behavior of numpy indeed hard understand... maybe else provide insight why does i not seek process arr in place - seems new array created under hood in cases anyway. i'd go arr = lut.take(arr) - way free half of memory used arr.

python numpy lookup-tables