Training convolutional neural networks on graphics...

Details Training convolutional neural networks on graphics... Training convolutional neural networks on graphics...

2005-08-31

2010-06-29

Ahmed, Samir A (Department: 2624)

Image analysis

Learning systems

Neural networks

C382S158000, C706S012000, C706S015000

Reexamination Certificate

active

07747070

ABSTRACT:
A convolutional neural network is implemented on a graphics processing unit. The network is then trained through a series of forward and backward passes, with convolutional kernels and bias matrices modified on each backward pass according to a gradient of an error function. The implementation takes advantage of parallel processing capabilities of pixel shader units on a GPU, and utilizes a set of start-to-finish formulas to program the computations on the pixel shaders. Input and output to the program is done through textures, and a multi-pass summation process is used when sums are needed across pixel shader unit registers.

REFERENCES:
patent: 5647027 (1997-07-01), Burges et al.
patent: 6128606 (2000-10-01), Bengio et al.
patent: 6553131 (2003-04-01), Neubauer et al.
patent: 2003/0174881 (2003-09-01), Simard et al.
patent: 2005/0025355 (2005-02-01), Simard et al.
patent: 2005/0091050 (2005-04-01), Surendran et al.
patent: 2005/0125369 (2005-06-01), Buck et al.
patent: 1345161 (2003-09-01), None
International Search Report from PCT/US2006/032349 dated Jan. 11, 2007, 3 pages.
Y. LeCun, L. Bottou, Y. Bengio, P. Ha.ner, “Gradient-based learning applied to document recognition,”Proceedings of IEEE, pp. 2278-2324, 1998.
Y. LeCun, Y. Bengio, “Convolutional networks for images, speech and time series,”The handbook of brain theory and neural networks, pp. 255-258, MIT Press, 1998.
P.Y. Simard, D. Steinkraus, J.C. Platt, “Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis,”Proc. International Conference on Document Analysis and Recognition, pp. 958-962, 2003.
Bishop, Christopher M., “Neural Networks for Pattern Recognition”, Chapter 4, Oxford University Press, 1995, pp. 116-163.
Anguita et al., “An Efficient Implementation of BP on RISC-Based Workstations,” Neurocomputing, vol. 6, No. 1, Feb. 1994, pp. 57-65.
Bohn, “Kohonen Feature Mapping Through Graphics Hardware,” 3rdIntl. Conf. on Computational Intelligence and Neurosciences, Oct. 24-28, 1998.
Chellapilla et al., “High Performance Convolutional Neural Networks for Document Processing,” Mar. 1, 2006, http://www.research.microsoft.com/en-us/um/people/kumarc/papers/chellapilla—iwfhr2006—ConvNNGPU.pdf.
European Search Report received in co-pending European Patent Application No. EP06801862, Feb. 17, 2010, 4 pages.
Goodnight et al., “Computation on Programmable Graphics Hardware,” Computer Graphics & Applications Magazine, vol. 25, No. 5, Aug. 26, 2005, pp. 12-15.
Oh et al., “GPU Implementation of Neural Networks,” Pattern Recognition, vol. 37, No. 6, Apr. 12, 2004, pp. 1311-1314.
Owens et al., “A Survey of General-Purpose Computation on Graphics Hardware,” Aug. 17, 2005, http://www.rudomin.cem.itesm.mx/{rudomin/Isaac%20Rudomin—files/cursos—files/GPGPU/ASurveyofGeneralPurposeComputationonGraphicsHardware.pdf.
Rolfes, “Artificial Neural Networks on Programmable Graphics Hardware,” Game Programming Gems 4, Mar. 2004, pp. 373-378.
Simard et al., “Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis,” Proceedings of the 7thIntl. Conf. on Document Analysis and Recognition, Aug. 3, 2003, pp. 958-963.
Steinkraus et al., “Using GPUs for Machine Learning Algorithms,” Proceedings of the 8thIntl Conf. on Document Analysis and Recognition, vol. 2, Aug. 29, 2005, pp. 1115-1120.

Affiliated with

Also associated with

Rating

Training convolutional neural networks on graphics... does not yet have a rating. At this time, there are no reviews or comments for this patent.

If you have personal experience with Training convolutional neural networks on graphics..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Training convolutional neural networks on graphics... will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFUS-PAI-O-4228346