Image analysis – Applications – Vehicle or traffic control
Reexamination Certificate
2001-02-02
2004-11-16
Mehta, Bhavesh M. (Department: 2625)
Image analysis
Applications
Vehicle or traffic control
C701S049000, C340S907000, C180S008300, C382S287000
Reexamination Certificate
active
06819780
ABSTRACT:
CROSS-REFERENCES TO RELATED APPLICATIONS
The present application is related to “A Method and Apparatus for Processing an Image of an Agricultural Field” filed by Benson, et al. on an even date herewith.
FIELD OF THE INVENTION
This invention relates generally to automatically steering agricultural vehicles in agricultural fields.
BACKGROUND OF THE INVENTION
With the advent of fast microprocessors, and inexpensive video cameras, researchers have recently begun to develop systems for guiding agricultural vehicles in agricultural fields. These systems are of particular benefit for cultivating equipment such as combines, harvesters, sprayers, rakes, harrows, or other implements. In most of these applications, the cultivating equipment needs to follow a very specific track through the field to efficiently cultivate the field. For example, when plowing a field the rows must be spaced a predetermined distance apart and should vary from adjacent rows by only a few inches. When spraying a crop, the sprayers traverse the field resting on their wheels, and must travel precisely between adjacent rows of previously planted crops. Again, this requires that the vehicle navigate a path that can vary only by a few inches. Harvesting crops also requires considerable accuracy to maintain the alignment between the harvesting head and the rows of crops that are harvested. Should the harvesting head drift more than a few inches to either side, it would begin to crush the crops rather than harvest them.
One of the things that makes automatic control of these processes particularly suitable was the presumed ability of video cameras to detect particular rows, spaces between rows, or the cut/uncut boundary of a field. Since the farmer, when manually guiding the vehicle, visually follows these features of the field, it is presumed that devising a camera that could extract this data and follow the same characteristics would be relatively easy. This has not proven to be the case.
Early attempts to use video cameras and image processors to receive and extract information indicative of a crop row, the space between rows or the cut/uncut boundary, were difficult to devise. Since crops vary widely in their height, their color, the “lushness” of growth, as well as their spacing, it was difficult to create a single method that would provide consistent edge or line detection of these features. Furthermore, agricultural fields during cultivation can be obscured by dust, dirt, flying plant matter or other objects straying into the field of view of the camera. All these problems have made the development of guidance and control systems quite difficult.
Even if the image processing and feature extraction (i.e., extracting the location of a crop row, the gap between crop rows, or the cut/uncut boundary) were possible, the accuracy of the guidance system did not prove sufficient to steer the vehicle within the few inches of permissible error.
In our previous applications entitled “A Method And Apparatus For Processing An Image Of An Agricultural Field” filed Jul. 10, 2000 and granted Ser. No. 09/612,688 and “A Method And Apparatus For Determining The Quality Of An Image Of An Agricultural Field Using A Plurality Of Fuzzy Logic Input Membership Functions”, filed on even date therewith, and granted Ser. No. 09/612,851, we described in some detail how the first of these problems could be solved. The second problem, that of controlling the location of the vehicle based upon the data extracted from each image frame, was not addressed.
In the present application, we describe the overall process of image processing and vehicle control in a manner that addresses both of the aforementioned difficulties.
SUMMARY OF THE INVENTION
In accordance with the first embodiment of the invention, a method of guiding an agricultural vehicle having at least one steerable wheel in an agricultural field having a plurality of rows of crops is provided, including the steps of taking a picture of the rows of crops with an electronic camera, successively processing a plurality of lines of the picture, extracting from the plurality of processed lines at least one value indicative of a crop line, extracting another value from the at least one value that is indicative of the lateral displacement of the crop line with respect to the vehicle, processing the another value in a fuzzy logic input membership function, then a fuzzy logic inference engine and then a fuzzy logic output membership function to develop a signal indicative of the degree the at least one steerable wheel should be turned, providing the signal indicative of the degree to which the steerable wheel should be turned to a feedback control circuit configured to steer the at least one steerable wheel in a direction that minimizes the difference between the signal indicative of the degree the at least one steerable wheel should be turned and another signal indicative of the degree to which the at least one steerable wheel is actually turned, generating a valve control signal calculated to reduce the difference, applying the valve control signal to an electro-hydraulic control valve configured to steer the at least one steerable wheel, and steering the wheel accordingly.
The step of extracting another value may include the step of extracting values sufficient to define a line indicative of a row of crops, a gap between a row of crops, or a cut/uncut boundary in a field. The step of processing the another value may include the step of processing the another value in the input membership function to produce at least a first input value indicative of a degree of membership in a first input membership function domain and a second input value indicative of a degree of membership in a second input membership domain. The step of processing the another value may include the step of processing the first and second input values in the fuzzy logic inference engine. The output membership function may include at least three domains symmetric with respect to a steering angle of zero. The at least three domains may include a center domain wherein a 100% membership causes the at least one steerable wheel to move toward or to remain in a mode of straight line travel. Two of the at least three domains may be symmetrically disposed on either side of the center domain. The at least three domains may include two additional domains that are symmetrically disposed on either side of a center of the center domain.
In accordance with a second embodiment of the invention, a method of guiding an agricultural vehicle having at least one steerable wheel in an agricultural field having a plurality of crops rows is provided included the steps of taking a picture of the crop rows with an electronic camera, extracting from the pictures row location values indicative of a location of at least one crop row, determining based upon at least one previous row location value whether the row location values are valid, and for each such row location values that are determined valid: deriving therefrom a value indicative of a desired direction of travel, processing the value indicative of a desired direction of travel in a fuzzy logic inference engine to produce a control signal, providing the control signal to an automatic control loop that uses feedback control based upon the position of the at least one steerable wheel to generate a valve command signal, applying the valve command signal to an electro-hydraulic steering control valve, and for each row location value that is determined invalid: not using such invalid row location value in the automatic control loop.
The step of deriving therefrom a value may include the step of processing the row location value in a first fuzzy logic control circuit. The value indicative of a desired direction of travel may be a value indicative of a desired steering angle or a desired steering rate. The step of determining whether the row location values are valid may include the step of processing the row location values in a second fuzzy logic control circuit. The row location values may includ
Benson Eric
Reid John F.
Zhang Qin
Azarian Seyed.
CNH America LLC
Henkel Rebecca
Maurer Brant T.
Mehta Bhavesh M.
LandOfFree
Method and apparatus for automatically steering a vehicle in... does not yet have a rating. At this time, there are no reviews or comments for this patent.
If you have personal experience with Method and apparatus for automatically steering a vehicle in..., we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Method and apparatus for automatically steering a vehicle in... will most certainly appreciate the feedback.
Profile ID: LFUS-PAI-O-3308638