I've got a bunch of images in a format similar to Cifar10 (binary file, size = 96*96*3
bytes per image), one image after another (STL-10 dataset). The file I'm opening has 138MB.
I tried to read & check the contents of the Tensors containing the images to be sure that the reading is done right, however I have two questions -
FixedLengthRecordReader
load the whole file, however just provide inputs one at a time? Since reading the first size
bytes should be relatively fast. However, the code takes about two minutes to run. sess.run(uint8image)
, however the result is empty.The code is below:
import tensorflow as tf
def read_stl10(filename_queue):
class STL10Record(object):
pass
result = STL10Record()
result.height = 96
result.width = 96
result.depth = 3
image_bytes = result.height * result.width * result.depth
record_bytes = image_bytes
reader = tf.FixedLengthRecordReader(record_bytes=record_bytes)
result.key, value = reader.read(filename_queue)
print value
record_bytes = tf.decode_raw(value, tf.uint8)
depth_major = tf.reshape(tf.slice(record_bytes, [0], [image_bytes]),
[result.depth, result.height, result.width])
result.uint8image = tf.transpose(depth_major, [1, 2, 0])
return result
# probably a hack since I should've provided a string tensor
filename_queue = tf.train.string_input_producer(['./data/train_X'])
image = read_stl10(filename_queue)
print image.uint8image
with tf.Session() as sess:
result = sess.run(image.uint8image)
print result, type(result)
Output:
Tensor("ReaderRead:1", shape=TensorShape([]), dtype=string)
Tensor("transpose:0", shape=TensorShape([Dimension(96), Dimension(96), Dimension(3)]), dtype=uint8)
I tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 4
I tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 4
[empty line for last print]
Process finished with exit code 137
I'm running this on my CPU, if that adds anything.
EDIT: I found the pure TensorFlow solution thanks to Rosa. Apparently, when using the string_input_producer
, in order to see the results, you need to initialize the queue runners.
The only required thing to add to the code above is the second line from below:
...
with tf.Session() as sess:
tf.train.start_queue_runners(sess=sess)
...
Afterwards, the image in the result
can be displayed with matplotlib.pyplot.imshow(result)
. I hope this helps someone. If you have any further questions, feel free to ask me or check the link in Rosa's answer.
This question is related to
python
tensorflow
I used CIFAR10 format instead of STL10 and code came out like
filename_queue = tf.train.string_input_producer(filenames)
read_input = read_cifar10(filename_queue)
with tf.Session() as sess:
tf.train.start_queue_runners(sess=sess)
result = sess.run(read_input.uint8image)
img = Image.fromarray(result, "RGB")
img.save('my.jpg')
The snippet is identical with mttk and Rosa Gronchi, but Somehow I wasn't able to show the image during run-time, so I saved as the JPG file.
According to the documentation you can decode JPEG/PNG images.
It should be something like this:
import tensorflow as tf
filenames = ['/image_dir/img.jpg']
filename_queue = tf.train.string_input_producer(filenames)
reader = tf.WholeFileReader()
key, value = reader.read(filename_queue)
images = tf.image.decode_jpeg(value, channels=3)
You can find a bit more info here
Load names with tf.train.match_filenames_once get the number of files to iterate over with tf.size open session and enjoy ;-)
import tensorflow as tf
import numpy as np
import matplotlib;
from PIL import Image
matplotlib.use('Agg')
import matplotlib.pyplot as plt
filenames = tf.train.match_filenames_once('./images/*.jpg')
count_num_files = tf.size(filenames)
filename_queue = tf.train.string_input_producer(filenames)
reader=tf.WholeFileReader()
key,value=reader.read(filename_queue)
img = tf.image.decode_jpeg(value)
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
num_files = sess.run(count_num_files)
for i in range(num_files):
image=img.eval()
print(image.shape)
Image.fromarray(np.asarray(image)).save('te.jpeg')
You can use tf.keras API.
import tensorflow as tf
import numpy as np
from tensorflow.keras.preprocessing.image import load_img, array_to_img
tf.enable_eager_execution()
img = load_img("example.png")
img = tf.convert_to_tensor(np.asarray(img))
image = tf.image.resize_images(img, (800, 800))
to_img = array_to_img(image)
to_img.show()
First of all scipy.misc.imread and PIL are no longer available. Instead use imageio library but you need to install Pillow for that as a dependancy
pip install Pillow imageio
Then use the following code to load the image and get the details about it.
import imageio
import tensorflow as tf
path = 'your_path_to_image' # '~/Downloads/image.png'
img = imageio.imread(path)
print(img.shape)
or
img_tf = tf.Variable(img)
print(img_tf.get_shape().as_list())
both work fine.
After speaking with you in the comments, I believe that you can just do this using numpy/scipy. The ideas is to read the image in the numpy
3d-array and feed it into the variable.
from scipy import misc
import tensorflow as tf
img = misc.imread('01.png')
print img.shape # (32, 32, 3)
img_tf = tf.Variable(img)
print img_tf.get_shape().as_list() # [32, 32, 3]
Then you can run your graph:
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
im = sess.run(img_tf)
and verify that it is the same:
import matplotlib.pyplot as plt
fig = plt.figure()
fig.add_subplot(1,2,1)
plt.imshow(im)
fig.add_subplot(1,2,2)
plt.imshow(img)
plt.show()
P.S. you mentioned: Since it's supposed to parallelize reading, it seems useful to know.
. To which I can say that rarely in data-analysis reading of the data is the bottleneck. Most of your time you will spend training your model.
(Can't comment, not enough reputation, but here is a modified version that worked for me)
To @HamedMP error about the No default session is registered
you can use InteractiveSession
to get rid of this error:
https://www.tensorflow.org/versions/r0.8/api_docs/python/client.html#InteractiveSession
And to @NumesSanguis issue with Image.show
, you can use the regular PIL .show()
method because fromarray
returns an image object.
I do both below (note I'm using JPEG instead of PNG):
import tensorflow as tf
import numpy as np
from PIL import Image
filename_queue = tf.train.string_input_producer(['my_img.jpg']) # list of files to read
reader = tf.WholeFileReader()
key, value = reader.read(filename_queue)
my_img = tf.image.decode_jpeg(value) # use png or jpg decoder based on your files.
init_op = tf.initialize_all_variables()
sess = tf.InteractiveSession()
with sess.as_default():
sess.run(init_op)
# Start populating the filename queue.
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
for i in range(1): #length of your filename list
image = my_img.eval() #here is your image Tensor :)
Image.fromarray(np.asarray(image)).show()
coord.request_stop()
coord.join(threads)
Source: Stackoverflow.com