Deep Learning by Ngene - Toolkit for LabVIEW Download

Deep Learning Toolkit for LabVIEW

2,897

Version	6.2.1.220
Released	Mar 30, 2023
Publisher	Ngene
License	Ngene Custom
LabVIEW Version	LabVIEW>=20.0
Operating System	Windows
Project links	Homepage Documentation Repository

Description

Empowering LabVIEW with Deep Learning
DeepLTK is a Deep Learning Toolkit for LabVIEW providing high-level API to build, configure, visualize, train, analyze and deploy Deep Neural Networks within LabVIEW. The toolkit is completely developed in LabVIEW and does not have any outer dependencies, which simplifies the installation, development, deployment and distribution of toolkit based applications and systems (particularly, can be easily deployed on NI's Real Time targets).

Main Features
Create, configure, train, and deploy deep neural networks (DNNs) in LabVIEW
Accelerate training and deployment of DNNs on GPUs
Save trained networks and load for deployment
Visualize network topology and common metrics (memory footprint, computational complexity)
Deploy pre-trained networks on NI's LabVIEW Real-Time target for inference
Speed up pre-trained networks by employing network graph optimization utilities
Analyze and evaluate network's performance
Start with ready-to-run real-world examples
Accelerate inference on FPGAs (with help of DeepLTK FPGA Add-on)

Supported Layers:
Input (1D, 3D)
Augmentations: Noise, Flip(Vertical, Horizontal), Brightness, Contrast, Hue, Saturation, Shear, Scale(Zoom), Blur, Move.
Fully Connected - FC
Convolutional - Conv2D
Convolutional Advanced - Conv2D_Adv
Upsampling
ShortCut (Residual)
Concatenation
Batch Normalization
Activations: Linear(None), Sigmoid, tanh(Hyperbolic Tangent), ReLU(Rectified Linear Unit), LReLU(Leaky ReLU)
Pooling (MaxPool, AvgPool, GlobalMax, GlobalAvg)
DropOut (1D, 3D)
SoftMax (1D, 3D)
YOLO_v2 (object detection)

Activation types:
Linear
Sigmoid
Hyperbolic Tangent
ReLU
Leaky ReLU
ReLU6
Mish
Swish

Solver (Optimization Algorithm):
Stochastic Gradient Descend (SGD) based Backpropagation algorithm with Momentum and Weight decay
Adam - Stochastic gradient descent method which is based on adaptive estimation of first-order and second-order moments.

Loss Functions:
MSE - Mean Squared Error
Cross Entropy (LogLoss)
Object Detection (YOLO_v2)

Examples:
Examples are available to demonstrate the applications of the toolkit in:
1. MNIST_Classifier_MLP.vi - training the deep neural network for image classification task in handwritten digit recognition problem (based on MNIST database) on 1 dimensional dataset using MLP (Multilayer Perceptron) architecture
2. MNIST_Classifier_CNN(Train).vi - training the deep neural network for image classification task in handwritten digit recognition problem using CNN (Convolutional Neural Network) architecture
3. MNIST_Classifier(Deploy).vi - deploying pretrained network by automatically loading network configuration and weights files generated from the examples above.
4. MNIST(RT_Deployment) project - deployment of pretrained model on NI's Real Time targets.
5. YOLO_Object_Detection(Cam).vi - automatically building and loading pretrained network for object detection based on YOLO (You Only Look Once) architecture.
6. Object_Detection project - demonstrates training of neural network for object detection on simple dataset.

Release Notes

6.2.1.220 (Mar 30, 2023)

v6.2.1
This is a minor update which does not break backward compatibility with v6.x.x versions of the toolkit.

Features
1.Added ReLU6 activation function.
2.Removed the requirement for using Conditional Disable Symbol ("NNG") for enabling GPU acceleration.
2.1.Removed GPU specific examples.

Bug Fixes
1.Fixed a bug in batch normalization.
2.Fixed a bug in the calculation of the MSE loss.
3.Fixed a bug in mAP metric calculations for object detection.
4.Increased the maximum number of layers in the network from 100 to 500.
5.Other minor bug fixes

Other Updates
1. Updated help file.
2. Added link to examples on GitHub in example instructions.

6.1.1.206 (Oct 18, 2022)

v6.1.1
This is a major update which does not break backward compatibility with v5.x.x version of the toolkit.

Features
1. Added support for Nvidia RTX 3xxx series of GPU by upgrading CUDA libraries.
2. Now CUDA libraries are part of the toolkit installer, which eliminates the need for separate installation of CUDA libraries.
3. Now all augmentation operations are accelerated on GPU, which greatly speeds up the training process when augmentations are enabled
4. Support for older versions of LabVIEW is deprecated. LabVIEW 2020 and newer are supported starting with this release.
5. Improved DeepLTK library loading time in LabVIEW.

5.1.2.190 (Jul 12, 2022)

v5.1.2
This is a minor update to the toolkit which aims to fix small bugs

Bug Fixes
1. Fixed issue related to toolkit's license activation.
2. Added missing descriptions in example VIs.
3. Fixed text format in VIP EULA.
4. Added missing text in help file.

v5.1.1
Backward Compatibility
Important: This is major update of the toolkit, which breaks backward compatibility with previous (pre v5.x.x) versions of the toolkit.
Features
1. Redesigned the process for specifying and configuring Loss Function. Now setting and configuring Loss function and configuring the training process are separated. New separate API for setting loss function (NN_Set_Loss.vi) is added.
2. Modified "NN_Train_Params.ctl".
a) Now loss function related parameters are removed from "NN_Train_Params.ctl".
b) "Momentum" is replaced with "Beta_1" and "Beta_2" parameters for specifying first and second order momentum coefficients.
c) "Weight_Decay" is replaced with "Weight_Decay(L1)" and "Weight_Decay(L2)" for specifyinh L1 and L2 weight normalization.
3. "NN_Eval_Test_Error_Loss.vi" is deprecated. Its functionality is now split between NN_Predict.vi" and "NN_Eval.vi"
4. Added support for Adam optimizer.
5. Added support for Swish and Mish activation functions.
6. Con3D layer is now renamed to Conv2D.
7. Added advanced Conv2D layer (Conv2D_Adv), which supports for:
a) dilation
b) grouped convolution
c) non square kernel window dimensions
d) non-square stride sizes
e) different vertical and horizontal padding sizes
8. Modified Upsample layer configuration control (Upsample_cfg.ctl) to separate vertical and horizontal strides.
9. Added new network performance evaluation API (NN_Eval.vi)
10. Label_Idx is removed from "NN_Predict.vi". Now classification predictions can be converted to categorical/binary labels with help of "NN_to_Categorical.vi"
11. Added new API for converting floating point predictions from network to categorical/binary (NN_to_Categorical.vi).
12. Added new API for categorical/binary labels to one-hot-encoded format (NN_to_OneHotEncoded.vi).
13. Now MaxPool and AvgPool layers support non square window dimensions.
14. Added new API control for 3D dimension representation (NN_Dims(C,H,W).ctl)
15. Region layer is now renamed to YOLO_v2.
a) Removed loss related configuration parameters (moved to NN_Set_Loss.vi configuration control).
b) Now anchor dimensions in YOLO_v2 layer should be provided in relative (to input image dimensions) format.
c) YOLO_v2 layer can automatically create last/preceding Conv2D layer, to match required number of classes and number of anchors.
16. Added support for Channel-Wise Cross-Entropy loss function for 3D output type networks with channel wise SoftMax output layer.
17. Added "Train?" control to "NN_Forward.vi" to take into account whether the network is in train state or not.
18. "NN_Calc_Confusion_Matrix.vi" is converted to polymorphic VI, which instance is chosen based on dataset provided at the input.
19. Optimized "NN_Draw_Rectangle.vi" for speed.
20. Increased Confusion Matrix table display precision from 3 to 4 digits.
21. Updated reference examples to make them compatible with latest changes.
22. Now DeepLTK supports CUDA v10.2 and CUDNN v7.5.x.
23. Configuration file format is updated to address feature changes.
24. Help file renamed to "DeepLTK_Help.chm"
25. Help file updated to represent recent changes.

Bug Fixes
1. Fixed a bug when MaxPool and AvgPool layers were incorrectly calculating output values on the edges.
2. Fixed a bug related to deployment licensing on RT targets.
3. Fixed a bug where receptive field calculation algorithm did not take into account dilation factor.
4. Corrected accuracy metrics calculation in "NN_Eval(In3D_OutBBox).vi".
5. Fixed typos in API VI descriptions and control/indicators.
6. Fix incorrect receptive field calculation for networks containing upsampling layer/s.
7. Fixed incorrect texts in error messages

4.1.1.166 (Sep 25, 2020)

v4.1.1.166
Features
1. Got rid of MASMT and TPLAT toolkit dependency to simplify installation process.
2. Added Possibility to use dataset's input feature maps in as ground truths to minimize the memory utilization when training autoencoder architectures.
3. Added new layer - Softmax3D to perform channel wise SoftMax operation over 3D type of feature maps/inputs.
4. Updated layer creation API to obtain layer's reference at creation.
5. Added possibility to calculate feature receptive field for each layer in the network.
6. Included network's output dimensions into NN.ctl.
7. Included Labels as string array in NN.ctl.
8. Updated the help documentation to reflect the changes.
9. Added API NN_Calc_Confusion_Matrix_Core.vi for inference.
10. Fixed a bug in BN Merge functionality.
Bugfixes
1. Updated Confusion Matrix table representation? Ground Truths - Row; Predictions - Columns.
2. Greatly sped up augmentation functions: up to 10x for some functions.
3. Fixed an improper operation when using some types of augmentation functions.
4. Fixed an improper operation when using pooling layer in global mode over non-square input features.
5. Fixed a string formatting in SVG network diagrams.
6. Fixed a bug for storing small numbers in .CFG file.
7. Made other bug fixes.

4.0.1.120 (Dec 23, 2019)

v4.0.1
Features
1.General performance improvements.
2.Added support for ShortCut (Residual) layer. Now ResNet architectures can be trained.
3.Added support for Concatenation layer.
4.Updated layer creation API to obtain layer's reference at creation.
5.Added API to calculate networks predictions over a dataset.
6.Added utility VI for Bounding Box format conversion.
7.Updated dataset data-type (cluster) to include file paths array of data samples.
8.Updated dataset data-type (cluster) to include labels as an array of strings.
9.Added possibility to set custom image dimensions (network's input resolution) when creating network topology from configuration file.
10.Added possibility to set custom mini batch size when creating network from configuration file.
11.Added utility VI to split large datasets into smaller portions (e.g. split training dataset into train and validation).
12.Added API to calculate and render confusion matrix based on networks predictions.
13.Added API to get detections over a batch of input samples.
14.Added API for mAP (mean Average Precision) evaluation for object detection tasks.
15.Added WarmUp feature into Learning Rate update policy.
16.Added API to get weights (values and references) from the layer.
17.Updated CUDA and CUDNN support to versions CUDA 10.1 an CUDNN 7.5.6.
18.Deprecated some configuration parameters in the layer creation API.
19.Updated examples to comply with the latest version of the toolkit.
20.Updated some API VI icons.
21.Changed data-flow wiring in SVG diagram for Shortcut, Concatenation layers and updated color scheme.
22.Deprecated Detection layer.
23.Speed up training and inference on GPU.
24.Added dependency requirements checking functionality during toolkit’s installation.
25.Fixed a bug preventing usage of more than 1 anchor boxes.
26.Fixed a bug caused "missing nng32.dll" in 32-bit version of LabVIEW.
27.Fixed a bug causing LabVIEW crash in LabVIEW 2017 and LabVIEW 2018.
28.Fixed bug causing LabVIEW crash when deploying networks with DropOut and/or DropOut3D layers.
29.Fixed bug rarely appearing when training network with LReLU activation for GPU.
30.Other bug fixes.

3.1.0.98 (Mar 29, 2019)

V3.1.0
Features
1. Added support for training with batch normalization.
2. Added utility to merge/fuse batch normalization into Conv3D or FC.
3. Added API to PULL/PUSH data from/to GPU.
4. Added utility to check GPU driver installation.
Bugfixes
1. Fixed issue with asynchronous computations on GPU.
2. Fixed dataset's size element type representation in In3D_Out3D dataset control.
3. Added missing API for dataset datatypes in front panel's function controls.
4. Fixed help links in the API.

3.0.1.93 (Feb 04, 2019)

V3.0.1
Features
1. Added support for training networks for object detection.
2. Added VIs for anchor box calculation based on annotated dataset.
3. Added VIs for calculating mAP (Mean Average Precision) for evaluating networks for object detection.
4. Added reference example for object detection.
5. Now when initializing weight number of first layers can be specified.
Suitable to transfer learning.
6. Added API to Set/Get DVR values (Polymorphic VIs for 1D, 2D, 3D, 4D, 5D arrays).
7. Added new type of dataset memory for object detection.
8. Added UpSample Layer.
9. Added support for online deployment license activation.
10. Updated help file to reflect the changes.
11. Updated API VIs description.

Bugfixes
1. Fixed GPU memory leakage issue.
2. Fixed deployment license issue.

2.0.1.76 (Aug 08, 2018)

v.2.0.1
1.Toolkit's name changed back from "DeepVIEW" to "Deep Learning Toolkit".

v.2.0.0
1.Added support for acceleration on GPUs.
2.Added GPU related examples.
3.Other bug fixes and performance improvements.
4.Restructured help document.
5.Added instructions for GPU toolkit installation.
6.Added description for new examples.
7.Updated GPU related API descriptions.

1.4.0.57 (Jun 14, 2018)

V1.4.0.57
1.Added augmentation functionality for 1-dimensional data implemented in Input1D layer.
2.Improved Conv3D and FC layer's performance.
3.Fixed augmentation errors in Input3D layer.

1.3.3.54 (May 31, 2018)

V1.3.3.54
1. Performance Improvements.
2. Removed Augmentation layer.
3. Added augmentation functionality into Input3D layer.
4. Added training support for datasets with different dimensionality:
a) 1-dimensional input -> 1-dimensional output
b) 3-dimensional input -> 1-dimensional output
c) 3-dimensional input -> 3-dimensional output
5. Added API for checking dataset compliance (i.e. input and output dimensions) with a built network
6. Improved Error Handling at SoftMax Layer creation.
7. Conv3D now supports DropOut3D as the input layer.
8. MaxPool and AvgPool layers now support non square inputs.
9. Added global MaxPool and AvgPool functionality.
10. Fixed metrics calculation for FC layer. Now number of params includes number of biases as well.
11. Improved Error Handling for checking dataset compliance with the built network.
12. Fixed a bug when writing a Region layer into configuration file.
13. YOLO Example. Now detected bounding boxes are provided in more convenient way for further processing.
14. YOLO Example. Now custom labels can be provided to be shown on the display.

1.3.2.51 (May 11, 2018)

V1.3.2.50
1. Performance Improvements.
2. Removed Augmentation layer.
3. Added augmentation functionality into Input3D layer.
4. Added training support for datasets with different dimensionality:
a) 1-dimensional input -> 1-dimensional output
b) 3-dimensional input -> 1-dimensional output
c) 3-dimensional input -> 3-dimensional output
5. Added API for checking dataset compliance (i.e. input and output dimensions) with a built network
6. Improved Error Handling at SoftMax Layer creation.
7. Conv3D now supports DropOut3D as the input layer.
8. MaxPool and AvgPool layers now support non square inputs.
9. Added global MaxPool and AvgPool functionality.
10. Fixed metrics calculation for FC layer. Now number of params includes number of biases as well.
11. Improved Error Handling for checking dataset compliance with the built network.
12. Fixed a bug when writing a Region layer into configuration file.
13. YOLO Example. Now detected bounding boxes are provided in more convenient way for further processing.
14. YOLO Example. Now custom labels can be provided to be shown on the display.

1.3.2.50 (May 05, 2018)

V1.3.2.50
1. Performance Improvements.
2. Removed Augmentation layer.
3. Added augmentation functionality into Input3D layer.
4. Added training support for datasets with different conventionalities:
a) 1-dimensional input -> 1-dimensional output
b) 3-dimensional input -> 1-dimensional output
c) 3-dimensional input -> 3-dimensional output
5. Added API for checking dataset compliance (i.e. input and output dimensions) with a built network
6. Improved Error Handling at SoftMax Layer creation.
7. Conv3D now supports DropOut3D as the input layer.
8. MaxPool and AvgPool layers now support non square inputs.
9. Added global MaPool and AvgPool functionality
10. Fixed metrics calculation for FC layer. Now number of params includes number of biases as well.
11. Improved Error Handling for checking dataset compliance with the built network.
12. Fixed a bug when writing a Region layer into configuration file

1.2.0.42 (Feb 08, 2018)

V1.2.0.41
1. Added support for deployment on NI’s RT targets.
2. Added API to get/set layer weights.
3. Added API to get layer outputs/activations.
4. Added API to get next layer.
5. Optimized weight initialization process.
6. Error Handling: Check input layer type at layer creation.
7. Error Handling: Check input dimensions at creating Conv3D layer.
8. Error Handling: Check input dimensions at creating Pool layer.
9. Error Handling: Check input data dimensions when setting Input3D layer outputs.
10. Fixed the bug for the case when neural network trained on non-square images.
11. Fixed error in get_next_layer.vi.
12. Added get layer data API.
13. Added warning at get layer data if not proper type of layer of routed at the input.
14. Error Handling: Check input dimensions at creating Conv3D layer.
15. Fixed a bug get new minibatch for dataset indexing when set for random sampling.
16. Updated instructions in MNIST examples.

V1.1.0.38
1. Added new examples - MNIST_Classifier_CNN(Train).vi, MNIST_Classifier(Deploy).vi. See description for more information
2. Updated help file
3. Fixed help file location in the installer
4. Corrected toolkit Vis' default values
5. Fixed bug when creating some layers (SoftMax, DropOut, DropOut3D) from configuration file.
6. Fixed errors generated by NN_Destroy.vi when empty network is provided at the input
7. Now probability set in DropOut layers will be coerced into (0..1) and warning will be generated if probability value set out of this range.
8. Fixed the issue of propagating warning through the toolkit VIs.
9. Added deployment licensing functionality
10. Included Ngene License Manager Installer in the toolkit installer
11. Other bugfixes

1.1.0.38 (Sep 05, 2017)

1. Added new examples - MNIST_Classifier_CNN(Train).vi, MNIST_Classifier(Deploy).vi. See description for more information
2. Updated help file
3. Fixed help file location in the installer
4. Corrected toolkit Vis' default values
5. Fixed bug when creating some layers (SoftMax, DropOut, DropOut3D) from configuration file.
6. Fixed errors generated by NN_Destroy.vi when empty network is provided at the input
7. Now probability set in DropOut layers will be coerced into (0..1) and warning will be generated if probability value set out of this range.
8. Fixed the issue of propagating warning through the toolkit VIs.
9. Added deployment licensing functionality
10. Included Ngene License Manager Installer in the toolkit installer
11. Other bugfixes

Show all release notes (+13 more releases)

Download Package

Versions

6.2.1.220 (Mar 30, 2023)

6.1.1.206 (Oct 18, 2022)

5.1.2.190 (Jul 12, 2022)

4.1.1.166 (Sep 25, 2020)

4.0.1.120 (Dec 23, 2019)

+ 9 more

Published by

Featured in

Post an Idea Post a Resource

Description

Release Notes

Recent Posts