I want to learn modern AI practical approaches from zero to implement it in my CAPCHA solving tasks.
AI BASICS
1. This approaches called as Machine learning or Neural network actually or even artificial neural network (ANN).
2. What solutions we have today. // scikit-learn (python), brain.js, ml5.js, TensorFlow github.com/tensorflow/tensorflow (Google), PyTorch, Theano or CNTK (Microsoft), Keras keras.io (works only on top of tensorflow or theano, MXNET, or CNTK), Chainer, NORB,
2.0. What models for image classification we have Inception V3 Model, VGG16, ConvNeXt, ResNe(X)t, DenseNet, EfficientNet, MobileNet,
2.0.1 Sound classification models YAMNet
2.1. What language generative models we have PaLM (Google), MakerSuite, LaMBDA, Bard (chatbot), BERT (used in google engine), Vertex AI, Open AI's Chat LGBT, Yandex LGPT.
2.2. Image gen AI. Vertex AI, ResNet, Inception, EfficientNet.
3. What terms could be used for object detection. // HOG and CNN (Convolutional Neural Networks)
4. What datasets we have. MNIST, CIFAR-10, IMDB, kaggle, movielens, Google Gmail Smartreply, Google Books Ngram, Palmer Pinguins, Movielens, Kaggle Dogs vs Cats, COCO captions, ImageNet (1.4M images). You could look for datasets at datasetsearch.research.google.com, GLUE, SST, horse or humans, Auto MPG, EuroSAT, Flickr8k, Wikimedia datasets (dumps.wikimedia.org), BooksCorpus, MEDLINE and PubMed, COLA (nyu-mll.github.io/CoLA/), tensorflow.org/datasets/catalog/
4.1 Audio datasets AudioSet-YouTube corpus, Mozilla’s Common Voice, TIDIGITS, CHiME-5, LibriSpeech, ESC-50, Maestro (Audio RNN)
Implement machine learning in SEO
1. Possibly I could gather articl'es parametrs and article's places in Search Engines to construct linear regression or maybe multiple regression or polynomial regression (not sure which is the best maybe last one) and predict parameters for first place at search, like:
from sklearn import linear_model
d = pandas.read_csv("data.csv")
X = d[['Weight', 'Volume']]
y = d['CO']
r = linear_model.LinearRegression()
r.fit(X, y)
predictCO = regr.predict([[2300, 1300]]) #predict
1.1 To verify if data actually correspond to regression use $ from sklearn.metrics import r2_score
PRETRAINED MODELS USAGE
1. To load pretrained models in Tensorflow use tf.keras.applications."model_name" and you could switch top level off by tf.keras.applications.ConvNeXtBase(include_top=False)
2. To use pretrained models in Keras use keras.applications."model_name"
3. Use models from kaggle.com/models and from github.com/tensorflow/models
4. For GLUE tasks you can use EXPERTS-BERT pretrained models.
TENSORFLOW
1. While importing tensorflow in pyCharm I've got an error: Process finished with exit code 132 (Interrupted by signal 4:SIGILL)
1.1. SIGILL means illigal assembly instruction probably for another architecture.
1.2. Try to delete second python interpreter. Probably I've downloade package for wrong architecture. // Nothing
1.3. Try to install with pip3 different version of tensorflow // $ pip3 install tensorflow==2.00
1.3.1. To check available versions $ pip3 index versions tensorflow
1.3.2. Install version 2.15.0, 2.14.1, 2.13.0, 2.12.0 // Same
2. How to create neural network with keras. Import
from tensorflow import keras
from tensorflow.keras import layers
2.1. Create 3 layers
import tensorflow as tf
m = tf.keras.Sequential()
m.add(tf.keras.layers.Dense(64,activation='relu') # relu adds unlinearity
m.add(tf.keras.layers.Dense(32,activation='relu')
m.add(tf.keras.layers.Dense(10)
or like:
model = tf.keras.models.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(10)
])
2.2. Compile network.
m.compile( optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'] )
2.2.1. Visualisation (optional)
vis = TensorBoard(log_dir="logs")
2.3. Train
m.fit(train_images, train_labels, epochs=10, callbacks=[vis])
2.4. To download MNIST
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
2.4.1 How to download MNIST on my machine. // You will see url in terminal
2.4.2. How to plot images from dataset
image = np.array(train_images[0], dtype='uint8')
plt.imshow(image)
plt.show()
2.5. To predict
probability_model = tf.keras.Sequential([model, tf.keras.layers.Softmax()])
predictions = probability_model.predict(list_my_new_images)
2.6. How to make tensor from image. load image file into python's matplotlib.
from matplotlib import image as mpimg
image = mpimg.imread("/home/mytest/5.png")
2.6.1. I have to make numpy.ndarray from image // It is not enough to use in tf
n = np.array(image)
2.6.2. Or try to import with //
f = open('fath/my_image.bmp', "rb")
data = np.fromfile(f, dtype=np.uint8) // Nothing
2.6.3. Try with keras
from keras.preprocessing.image import load_img
from keras.preprocessing.image import img_to_array
from keras.preprocessing.image import array_to_img
img = load_img('/home/mytest/5.png')
img_numpy_array = img_to_array(img) # convert the image into numpy array // Nothing
2.6.4. Try
data_dir = pathlib.Path("/home/AI/mytest/")
val_ds = tf.keras.utils.image_dataset_from_directory(
data_dir,
validation_split=0.2,
subset="validation",
seed=123,
image_size=(img_height, img_width),
batch_size=batch_size)
// I've got error No images found in directory /home/john/Downloads/AI/mytest/. Allowed formats: ('.bmp', '.gif', '.jpeg', '.jpg', '.png')
2.6.5. Convert image to tensor Tensorflow. Create image bmp black&white (8-bit grayscale) vertical images each 28x28
from PIL import Image
import tensorflow as tf
img = Image.open('path/my_image.jpg')
t = tf.convert_to_tensor(img, dtype=tf.float32)
t2 = tf.reshape(t, [1,28,28])
t2 = t2 / 255
t2 = t2 - 1
// But it is half solution i want to use colored images
2.6.6. How to implement mask.
2.6.7. How to import image with target size (like 28x28)
from keras.preprocessing import image
img = image.load_img("/home/picture.bmp", color_mode="grayscale", target_size=(28,28))
2.6.8. Reshape ndarray from shape 28x28x1 to 28x28. // new_arr = np.reshape(old_arr, (28,28))
2.6.9. How to invert colors ndarray // numpy.invert(img)
image = 255 - image
2.6.10. Alternative method to predict image from folder
import PIL.Image as Image
im = Image.open('/home/Datasets/saves/transf/ostrich.jpg').resize((200,200))
im = np.array(im)/255.0
result = model.predict(im[np.newaxis, ...])
2.6.10.1. Import an image with tensorflow only.
path = "/home/path/img_cat.jpg"
image = tf.io.read_file(path)
img = tf.io.decode_jpeg(image, channels=1)
img_r = tf.image.resize(img, size=[333,333])
im = tf.image.convert_image_dtype(img_r, dtype=tf.float32)
im = tf.expand_dims(im, axis=0)
2.6.11. To make grayscale:
gray = tf.image.rgb_to_grayscale(img)
2.7. CNN Convolution neural network may help to find some object in unknown position at image. In keras it Conv2D layer.
2.7.1. Convolution technic somehow helps to detect common patterns of image like shape or edges or background. It uses different filters (kernels).
For example filter below helps to pass sharp edges:
[[0,1,0],
[1,-4,1],
[0,1,0]]
Filter for vertical lines:
[[-1,0,1],
[-2,0,2],
[-1,0,1]]
Filter for horizontal lines:
[[-1,-2,-1],
[0,0,0],
[1,2,1]]
Ridge and edge detection:
[[-1,-1,-1],
[-1,8,-1],
[-1,-1,-1]]
Sharpen:
[[0,-1,0],
[-1,5,-1],
[0,-1,0]]
Blur: // Doesn't work
[[1,1,1],
[1,1,1],
[1,1,1]]
Gaussian blur: // Doesn't work
[[1,2,1],
[2,4,2],
[1,2,1]]
Emboss:
[[-2,-1, 0],
[-1, 1, 1],
[ 0, 1, 2]]
2.7.2. Along with convolutions you have to use pooling technic. It is kind of increase contrast of the convoluted image.
2.7.3. CAM technic also could be helpfull for recognizing oblects at different parts of an image.
2.8. To generate more images (rotate, zoom, flip) for dataset use augmantation.
images = ImageDataGenerator(
rescale = 1./255,
rotation_range = 40,
width_shift_range = 0.2,
height_shift_range = 0.2,
shear_range = 0.2,
horizontal_flip=True,
fill_mode='nearest')
2.9. To use pretrained model you have to freeze old layers and train only last new few layers. It is called fine-tuning feature extraction or transfer learning (fine tuning it is when you little bit retrain old model with very low learning rate and small amount of epochs. To load models.
import tensorflow_hub as hub
hub_model = hub.load(model)
3. To detect objects on image use TensorFlow Object Detection API
4. Datasets you downloaded with tfds.load() are saved in the local folder /home/john/tensorflow_datasets/t
5. Use Active Lerning model principle.
6. Use Committe sampling or commette based paradigm. When you get decision based on many models.
TRANSLATION
4. For translation use NMT approaches (Neural machine translation)
4.1. You need language datasets like Anki.
4.2. For translation best choise is Transformer neural network layer.
5. While running transformer.summary() on tensorflow model I get an error ValueError: This model has not yet been built. Build the model first by calling `build()` or by calling the model on a batch of data.
5.1. Try to run a test on example of data before transformer.summary():
for (pt, en), en_labels in train_batches.take(1):
break
output = transformer((pt, en))
PERFORMANCE ISSUES. MAKE FITTING FASTER
1. Usage of model.compile(jit_compile = True) make it faster for tensorflow.
2. I get a problem. While I am trying to fit my model in Tensorflow it takes to mach CPU and never reach the end.
3. Use float16 data rather then float32, It executed faster.
4. The most simple way to decrease fit time is to decrease number of layers and number of units in layers.
5. While saving my model got an error NotImplementedError: Learning rate schedule 'CustomSchedule' must override `get_config()` in order to be serializable.
5.1. Try to set optimizer = 'adam' // Still get WARNING:tensorflow:Model's `__init__()` arguments contain non-serializable objects. Please implement a `get_config()` method in the subclassed Model for proper saving and loading. Defaulting to empty config. But model has been saved but when I load it I get ValueError: Unable to restore custom object of class "MeanMetricWrapper" (type _tf_keras_metric). Please make sure that this class is included in the `custom_objects` arg when calling `load_model()`. Also, check that the class implements `get_config` and `from_config`.
5.1.1. Serializable means convertable
5.1.2. Seems to be it is inpossible to save keras model with subclasses. Delete classes or try to save weights.
5.2. Try transformer.save_weights('path/filename.cpkt') then transformer.load_weights('path/filename.cpkt')
MY THOUGHTS
1. Difference between neural net and human brain. In human brain not only weights are changing during learning but the model structure.