Omnivision OV7640 YUV data structure clarification

[Paul Olley]

Hello,

When the OV7640 is reset, the registers have their default values.

The ones that interest me are :

COMA (at address 12h) ; default value 14h
COMB (at address 13h) ; default value A3h
COMD (at address 15h) ; default value 00h

As a result of these settings, the datasheet specifies the data coming out
of Y[7:0] is structured as follows : UYVYUYVY

On the front page of the datasheet a feature is listed as YUV (4:2:2)

I understand the 4:2:2 means that Y is sampled at twice the frequency of U
and V, which is why Y appears twice as frequently as U or V in the extract :
UYVYUYVY

I would like to clarify my understanding of the data I capture :

For each frame, I capture 640 x 480 pixels = 307200 pixels

UYVYUYVY

Pixel 1 : is a U value
Pixel 2 : is a Y value
Pixel 3 : is a V value
Pixel 4 : is a Y value
....

the first pixel of line 2 : Pixel 641 : is a U value ?
Pixel 642 : is a Y value ?
Pixel 643 : is a V value ?
Pixel 644 : is a Y value ?

For example, for pixel 642, I have the Y value, but no corresponding U or V.
So in order to estimate the U or V value for this pixel, I need to use an
algorithm such as nearest neighbor replication or bilinear interpolation?

Is this understanding correct? Or is the data packed in a similar way to
RGB 5:6:5 where within 2 bytes of Y[7:0] you get an R,G and a B value
(albeit not at 8 bits resolution!)

Thank you for your assistance.

Paul.

 

[Paul Carpenter]

Hello,

When the OV7640 is reset, the registers have their default values.

The ones that interest me are :

COMA (at address 12h) ; default value 14h
COMB (at address 13h) ; default value A3h
COMD (at address 15h) ; default value 00h

As a result of these settings, the datasheet specifies the data coming out
of Y[7:0] is structured as follows : UYVYUYVY

On the front page of the datasheet a feature is listed as YUV (4:2:2)

I understand the 4:2:2 means that Y is sampled at twice the frequency of U
and V, which is why Y appears twice as frequently as U or V in the extract :
UYVYUYVY

Alternatively look at this way for every TWO Y pixels, there is only ONE
corresponding U and ONE corresponding V. You have to decode the colour
matrix of U and V to use with TWO adjacent Y points. U used with one pair
of Y and V used with a different pair (ideally).

I would like to clarify my understanding of the data I capture :

For each frame, I capture 640 x 480 pixels = 307200 pixels

UYVYUYVY

Pixel 1 : is a U value
Pixel 2 : is a Y value
Pixel 3 : is a V value
Pixel 4 : is a Y value
...

No that is four data points and TWO pixels.

the first pixel of line 2 : Pixel 641 : is a U value ?
Pixel 642 : is a Y value ?
Pixel 643 : is a V value ?
Pixel 644 : is a Y value ?

I would expect the first data BYTE of line 2 to be data BYTE 1281 (starting
at 1 or 1280 starting at 0).

For example, for pixel 642, I have the Y value, but no corresponding U or V.
So in order to estimate the U or V value for this pixel, I need to use an
algorithm such as nearest neighbor replication or bilinear interpolation?

The bytes coming out should be in multiples of FOUR to give 4:2:2 coding,
which correspond in CCIR656 format (4:2:2 down one byte wide stream) as
twice as many data points as pixels.

Is this understanding correct? Or is the data packed in a similar way to
RGB 5:6:5 where within 2 bytes of Y[7:0] you get an R,G and a B value
(albeit not at 8 bits resolution!)

You really need FOUR bytes to obtain an RGB matrix conversion as 4:2:2 should
be sampled as

Y Y Y
U U
V V

Hence the pairings of U and V for diffeent pairs of Y. Until you have the
THIRD byte at least you cannot give an RGB value for the first PIXEL.

However they are often sampled for simpler clocking and data pipeline

Y Y Y Y
U U
V V

Until you have the second pair of bytes the RGB values cannot be determined.

A good reference is Keith Jack's Video Demystified (now edition 4)
with a link from the Omnivision technical pages.

ISBN 0-7506-7822-4

Analog to IPTV described and algorithms and conversion matrices.

Just gone from 1st edition to 4th edition myself.

 

[Paul Olley]

Hello Paul,

Many thanks for your quick, detailed response.

I have the Omnivision datasheet which is marked (on the front page) Verson
1.7, October 27, 2003.

Do you have access to this version?

I want to start by looking at how many pixels I capture first, since I will
be going nowhere if I've got this wrong.

For each frame, I capture 640 x 480 pixels = 307200 pixels

On page 9 of the datasheet, there is a FIGURE 8 : VGA Frame Timing Diagram.

This shows each row to consist of 640 x t PCLK (ie : not 1280)

Have I misunderstood that YUV data will be output with the timing of Figure
8?

I would expect the first data BYTE of line 2 to be data BYTE 1281
(starting
at 1 or 1280 starting at 0).

At this point I have a theory.

My "system" clock is 20MHz, and PCLK is therefore running at 10MHz
(according to the OV datasheet).

I use a CPLD to do the following :

The microcontroller asks the CPLD to capture a frame.
The CPLD waits for VSYNC to go high and then low. (Now we are ready to
capture a frame, the next HREF will indicate the first row to capture)
*
If VSYNC goes high, then exit the capture sequence and tell microcontroller
the capture sequence is complete.
If HREF is high, then Y[7:0] is written into a SRAM and the address counter
incremented by 1.
Goto *

I then transfer all 307200 pixels to a PC via RS232.

I assign a luminance only value based on the byte value at that location,
and display it as 640 x 480.

What I see is a monochrome image, each pixel appears logically in the
correct place, by which I mean that there isn't any wrap around.

If the OV was outputting twice as many pixels as I was capturing, the only
way that would be possible is if I was recording every other pixel and PCLK
would therefore have to be running twice as fast as I think it is.
(According to the datasheet.)

However, I have an OV Application note, which shows a formula for PCLK, such
that you can NEVER achieve PCLK running faster than half of OV external
CLOCK.

I will go and measure PCLK first and confirm its frequency.

A good reference is Keith Jack's Video Demystified (now edition 4)
with a link from the Omnivision technical pages.

Thank you for this recommendation. I will purchase this very soon.

Paul.

 

[Paul Carpenter]

Hello Paul,

Many thanks for your quick, detailed response.

I have the Omnivision datasheet which is marked (on the front page) Verson
1.7, October 27, 2003.

Do you have access to this version?

Yes and I have been looking at it. It has many things missing notably YUV
format data streams.

I want to start by looking at how many pixels I capture first, since I will
be going nowhere if I've got this wrong.

How do you determine capture?

Counter counting the first 640 falling edges of PCLK? (as this is determined
by COMD bit 6 as the reference edge for Y[7:0}). When you run out of data
space what happens to other data is it lost.

Please also consider that to meet 4:2:2 data stream for there must 1280
bytes of 8 bit data for 640 pixels in 8bit 4:2:2 format.

Please note even though a frame of 640 x 480 pixels = 307200 pixels

If you are transmitting a frame of 16 bit values as 4:2:2 this requires

307200 * 2 = 614400 BYTES of storage

307200 BYTES is only enough to save a MONOchrome image of
eight bits per pixel.

On page 9 of the datasheet, there is a FIGURE 8 : VGA Frame Timing Diagram.

This shows each row to consist of 640 x t PCLK (ie : not 1280)

Have I misunderstood that YUV data will be output with the timing of Figure
8?

Yes and NO.

Firstly PCLK is the Pixel NOT byte clock in frequency.

Secondly this is a datasheet for a MONO and colour device and preliminary
at that. Not uncommon from Omnivision.

Look at the Block diagram AND the A/D description, these A/D converters
(note there are TWO) are limited to 12MHz speed, hence PCLK must never exceed
CLK / 2 (which is limited to 20MHz).

So at PCLK rate TWO A/D converters convert values to be passed to the
Output formatter to be passed using default values down one 8 bit data path.
The default configuration is YUV as 8 bit data (1-254 as 0 and 255 are
reserved). So two values are created every Pixel clock but where has the
second A/D conversion gone?

Now this is why I separate the terms BYTEs and PIXELS. COMD bit 6 is a
big clue to me that if you were to check with a scope to see if you
have STABLE data on the rising and falling edges of PCLK, that is
where your colour data is.

I would expect the first data BYTE of line 2 to be data BYTE 1281
(starting
at 1 or 1280 starting at 0).

At this point I have a theory.

My "system" clock is 20MHz, and PCLK is therefore running at 10MHz
(according to the OV datasheet).

I use a CPLD to do the following :

The microcontroller asks the CPLD to capture a frame.
The CPLD waits for VSYNC to go high and then low. (Now we are ready to
capture a frame, the next HREF will indicate the first row to capture)
*
If VSYNC goes high, then exit the capture sequence and tell microcontroller
the capture sequence is complete.
If HREF is high, then Y[7:0] is written into a SRAM and the address counter
incremented by 1.

What are you doing about the OTHER clock edge and its data.
The data must be there.

What happens when HREF goes low?

If you provide scope outputs from the CPLD what can you see about start
and stop of captures compared to PCLK and HREF?

Goto *

I then transfer all 307200 pixels to a PC via RS232.

What happened to the other half of the data? How do you determine if you
have the full data?

What happens to the data if more than 307200 BYTES are received, if this is
exactly HALF an image how do you know the image is the full width and height
of what is being captured?

307200 is only enough bytes for a MONOchrome image.

I assign a luminance only value based on the byte value at that location,
and display it as 640 x 480.

Bad move displaying an image, unless it is of a WELL illuminated test chart
will tell you very little as:-

a) If by chance you are only capturing the first 640 bytes for each line
How do you know that data is the FULL width of what the device is
capturing?

b) If the target image is not very colourful (the vast majority of views)
Chrominance data is offset by 128 as it is SIGNED values, this can give
every second pixel a grey mid range value. You would be amazed how little
range U and V have on a lot of video sources, even from a camera.

c) You are not capturing the colour data and have not made provisions for it.

Put some form of test chart in front of the camera (even coloured rainbow
ribbon cable - the poor man's colour bars test chart), know what you should
be seeing. Look at the DATA from the first byte onwards to see if it makes
sense. Even for a black and white target the U and V data should be
around 128 for every second BYTE for no colour.

What I see is a monochrome image, each pixel appears logically in the
correct place, by which I mean that there isn't any wrap around.

Either as I suspect you are not capturing the colour data, or the image is
not effected by every second pixel being mid grey and only half the line of
data.

If the OV was outputting twice as many pixels as I was capturing, the only
way that would be possible is if I was recording every other pixel and PCLK
would therefore have to be running twice as fast as I think it is.
(According to the datasheet.)

Alternatively the colour data is output on the other edge.

However, I have an OV Application note, which shows a formula for PCLK, such
that you can NEVER achieve PCLK running faster than half of OV external
CLOCK.

I will go and measure PCLK first and confirm its frequency.

Check what happens over a couple of PCLK cycles for Y7 and Y0 for BOTH edges
of the clock.

To support 4:2:2 as 8 bit Y, U and V each and ITU 656 it MUST be going at
twice the Pixel rate, that is a function of 4:2:2 in 656 format. This format
is not fully described in their data sheet and their diagrams for RGB 555
and RGB 565 leave a lot to be desired as this suggests half the H resolution
is provided. I suspect this is what they refer to as 'preliminary'.

I really wish they provided a VBS signal from this device (Video, Blanking,
and Sync) to take the Y signal to some form of other monitor to compare
the data captured.

 

[Paul Olley]

Hello Paul,

Yes and I have been looking at it. It has many things missing notably YUV
format data streams.

Indeed. On a side issue, unless I am having a simple problem navigating
their site, have you noticed that Omnivision have removed datasheets for
their sensors from their website recently? Then curiously they leave the
datasheet for the SCCB bus on their site. Or am I just not looking in the
right place?

So reading your reply I immediately see my problem. Not having experience
of video, I designed my hardware around the VGA Frame Timing Diagram,
concluding that I would need to store a maximum of 307200 bytes of
information per frame. On this bases I have a 512k SRAM. I do store
exactly 307200 bytes, and for interest sake I will explain the control logic
in the CPLD.

First, I want to examine what data I am actually capturing. If two bytes of
valid data are available per cycle of PCLK (one on each edge), then I am
capturing the data only on one edge. (ie : half the available data) I
suspect that I am capturing on the Y values, as I get get a very good
monochrome image. Whilst this makes sense for one line, if I understand
correctly, the Y values won't appear in the same position on the second
line. ie : line 1 byte 2 (not pixel 2) is Y but line 2 byte 2 won't be a Y.
I won't waste too much time thinking about this, as what really matters is
that I capture all the bytes correctly.

Back to the control logic of the CPLD.

Remember here, that I am capturing 307200 bytes. In effect, I "gate" the
SRAM such that it stores 640 samples during the time that HREF is active.

How do you determine capture?

Counter counting the first 640 falling edges of PCLK? (as this is
determined
by COMD bit 6 as the reference edge for Y[7:0}). When you run out of data
space what happens to other data is it lost.

The CPLD clock is running at twice PCLK, which in two cycles gives me time
to increment the address counter and strobe !WE.

Actually, I don't look at PCLK at all. I first wait for VSYNC. After VSYNC
has gone low, I know the next time HREF goes high, it will be the first line
of the frame.

Then I wait for HREF to go high. Whilst it is high, I alternately strobe
!WE and increment the address counter. After each byte write, I check if
HREF is still high, if it is, I perform another write cycle. If it has gone
low, I wait for two possible events. (And this I think answers your
question If I see HREF go high again, it is the start of a new line and I
start writing bytes into SRAM. If on the other hand I don't see HREF go
high, but instead see VSYNC go high, then clearly it is the start of a new
frame, and I have captured all the information relevant to one frame. At
this point I conclude the capture sequence is complete. Since I am
interested in capturing a still frame, not video, this simple approach seems
to work fine.

To capture on both edges of PCLK, I will need to (a) Reduce the PCLK rate by
half, and (b) increase the SRAM size from 512k to 600k

The SRAM size is now an issue. Whilst 512k is reasonably priced, any larger
and I think my approach should change.

Remaining production lifetime of the OV7640 may be an issue, and while I'm
at it, why not make provision for an OV9650 (1.3 M pixels) I see at
2k1.co.uk that the OV9650 is less expensive than the OV7640! I don't
recognize the package outline (the limited information I can see on this
device), and I wonder, can I even hand assemble the OV9650? I'm waiting for
a response to a request for the datasheet for it.

The next issue will be RAM, so maybe it is time to look at a (less
expensive) DRAM. I would need 1300 x 1028 x 2(!) = 2672800 bytes. This is
only £3.24 + VAT at Farnell, but of course the (CPLD) controller complexity
is much higher.

Firstly PCLK is the Pixel NOT byte clock in frequency.

OK. I will try and scope PCLK and Y[7:0] to see if I get valid data on both
edges. Since I don't have a DSO, this might be a little bit of a challenge,
but I'll soon see.

Secondly this is a datasheet for a MONO and colour device and preliminary
at that. Not uncommon from Omnivision.

Yes. I never found an OV7640 datasheet that wasn't "preliminary"

Now this is why I separate the terms BYTEs and PIXELS. COMD bit 6 is a
big clue to me that if you were to check with a scope to see if you
have STABLE data on the rising and falling edges of PCLK, that is
where your colour data is.

Thank you for this explanation.

Bad move displaying an image, unless it is of a WELL illuminated test
chart
will tell you very little as:-

a) If by chance you are only capturing the first 640 bytes for each line
How do you know that data is the FULL width of what the device is
capturing?

I believe I am capturing every other byte of a line, ie : 640 bytes when in
fact there are 1280 "available" for capture.

b) If the target image is not very colourful (the vast majority of views)
Chrominance data is offset by 128 as it is SIGNED values, this can give
every second pixel a grey mid range value. You would be amazed how
little
range U and V have on a lot of video sources, even from a camera.

OK. In fact, I may already have seen this, whilst playing with the
registers and selecting different output formats, I captured data whilst was
grey and had VERY little variation in values. I think this was due to
shifting the format from UYVY to YUYV which makes sense. I was capturing
the colour information, but not the luminance.

c) You are not capturing the colour data and have not made provisions for
it.

This is exactly the conclusion I draw thanks to your explanation.

Put some form of test chart in front of the camera (even coloured rainbow
ribbon cable - the poor man's colour bars test chart), know what you
should
be seeing. Look at the DATA from the first byte onwards to see if it makes
sense. Even for a black and white target the U and V data should be
around 128 for every second BYTE for no colour.

I will modify the logic as follows :

Slow PCLK down so I can capture every BYTE of output. I will exceed the
store capacity of the SRAM by doing this, so I will come up with a plan to
capture say half the number of vertical lines. This will give me the
opportunity of checking that I capture the output correctly. (For at least
half the frame anyway.)

I can then work on a new memory design sufficient to capture all bytes in a
DRAM.

To support 4:2:2 as 8 bit Y, U and V each and ITU 656 it MUST be going at
twice the Pixel rate, that is a function of 4:2:2 in 656 format. This
format
is not fully described in their data sheet and their diagrams for RGB 555
and RGB 565 leave a lot to be desired as this suggests half the H
resolution
is provided. I suspect this is what they refer to as 'preliminary'.

Pity the datasheet wasn't more accurate :-(

I really wish they provided a VBS signal from this device (Video,
Blanking,
and Sync) to take the Y signal to some form of other monitor to compare
the data captured.

Not that I've looked closely, but don't some other OV devices have a
separate Y channel (ie : 8 pins for Y and 8 pins for UV)?

It is a pity that devices such as OV519 are not available in small
quantities. I saw another device, a Sanyo LC82210 that would make image
capture a breeze. (I saw a simple block diagram here
http://www.alldatasheet.co.kr/datasheet-pdf/pdf_kor/SANYO/LC82210.html but
this part is not readily available.

Still, maybe it is more fun doing it the hard way, and then the only supply
problem becomes the lifetime of the OV part.

Paul.

 

[Paul Carpenter]

On Thursday, in article
<[email protected]>

Hello Paul,


Indeed. On a side issue, unless I am having a simple problem navigating
their site, have you noticed that Omnivision have removed datasheets for
their sensors from their website recently? Then curiously they leave the
datasheet for the SCCB bus on their site. Or am I just not looking in the
right place?

I had to get their data sheets from another site.

So reading your reply I immediately see my problem. Not having experience
of video, I designed my hardware around the VGA Frame Timing Diagram,

That is the issue for a lot of people video is not teh same as graphics cards
and those dealing with graphics think they have problems with resolutions.
Just wait till you have to deal with all the video formats!

concluding that I would need to store a maximum of 307200 bytes of
information per frame. On this bases I have a 512k SRAM. I do store
exactly 307200 bytes, and for interest sake I will explain the control logic
in the CPLD.

First, I want to examine what data I am actually capturing. If two bytes of
valid data are available per cycle of PCLK (one on each edge), then I am
capturing the data only on one edge. (ie : half the available data) I
suspect that I am capturing on the Y values, as I get get a very good
monochrome image. Whilst this makes sense for one line, if I understand
correctly, the Y values won't appear in the same position on the second
line. ie : line 1 byte 2 (not pixel 2) is Y but line 2 byte 2 won't be a Y.
I won't waste too much time thinking about this, as what really matters is
that I capture all the bytes correctly.

I think the byte order is swapped on successive lines just like PAL alternates
its colour phase, however the same edge determines which byte it is! So the
starting phase may be different.

Back to the control logic of the CPLD.

Remember here, that I am capturing 307200 bytes. In effect, I "gate" the
SRAM such that it stores 640 samples during the time that HREF is active.

How do you determine capture?

Counter counting the first 640 falling edges of PCLK? (as this is
determined
by COMD bit 6 as the reference edge for Y[7:0}). When you run out of data
space what happens to other data is it lost.

The CPLD clock is running at twice PCLK, which in two cycles gives me time
to increment the address counter and strobe !WE.

Actually, I don't look at PCLK at all. I first wait for VSYNC. After VSYNC
has gone low, I know the next time HREF goes high, it will be the first line
of the frame.

Then I wait for HREF to go high. Whilst it is high, I alternately strobe
!WE and increment the address counter. After each byte write, I check if
HREF is still high, if it is, I perform another write cycle. If it has gone ....
To capture on both edges of PCLK, I will need to (a) Reduce the PCLK rate by
half, and (b) increase the SRAM size from 512k to 600k

Alternatively don't route the data through the PLD use TWO external latches
to latch the data (16bit wide) on each phase, use nearly the same timing in the
CPLD and write to TWO RAM chips at once from the outputs of the latches.

The SRAM size is now an issue. Whilst 512k is reasonably priced, any larger
and I think my approach should change.

Or two RAM chips or Field stores as you write in and read out sequentially.
Field store chips are quite cheap.

Remaining production lifetime of the OV7640 may be an issue, and while I'm
at it, why not make provision for an OV9650 (1.3 M pixels) I see at
2k1.co.uk that the OV9650 is less expensive than the OV7640! I don't
recognize the package outline (the limited information I can see on this
device), and I wonder, can I even hand assemble the OV9650? I'm waiting for
a response to a request for the datasheet for it.

This is a chip scale package, primarily meant for chip on board hybrids for
'throwaway' devices like toys, webcams and mobile phones. These devices are
made in bulk as one batch maybe two batches, sold and forgotten about.

The next issue will be RAM, so maybe it is time to look at a (less
expensive) DRAM. I would need 1300 x 1028 x 2(!) = 2672800 bytes. This is
only ?3.24 + VAT at Farnell, but of course the (CPLD) controller complexity
is much higher.

Alternatively instead of reinventing the wheel use one of the prebuilt
serial cameras that already have the guts in to do JPEG compression from
an Omnivision sensor.

See http://home.pacific.net.hk/~comedia/c328_7640.htm
Available from www.roundsolutions.co.uk very cheaply
Starter kit of something like 2 cameras, adapters, software
for about 132 pounds UK ! STK-CAM-VGA100, buy cameras in at
least threes for about 30 pounds each!

or the Intertec one from TDC

http://www.tdc.co.uk/camera/index.htm#itmc328

Firstly PCLK is the Pixel NOT byte clock in frequency.

OK. I will try and scope PCLK and Y[7:0] to see if I get valid data on both
edges. Since I don't have a DSO, this might be a little bit of a challenge,
but I'll soon see.

Delayed triggering of an analog scope should show whether you have valid data
at those times.

....

I believe I am capturing every other byte of a line, ie : 640 bytes when in
fact there are 1280 "available" for capture.


OK. In fact, I may already have seen this, whilst playing with the
registers and selecting different output formats, I captured data whilst was
grey and had VERY little variation in values. I think this was due to
shifting the format from UYVY to YUYV which makes sense. I was capturing
the colour information, but not the luminance.

Sounds like you changed the Y clock edge to oposite polarity.

....

I will modify the logic as follows :

Slow PCLK down so I can capture every BYTE of output. I will exceed the
store capacity of the SRAM by doing this, so I will come up with a plan to
capture say half the number of vertical lines. This will give me the
opportunity of checking that I capture the output correctly. (For at least
half the frame anyway.)

Consider piggybacking a latch and second RAM for now as 16 bit write from PLD
and latch.

Alternatively buy in cheap serial cameras.

I can then work on a new memory design sufficient to capture all bytes in a
DRAM.

Look at Field/Frame memories as these are cheap memories for video recorders
and the like.

Not that I've looked closely, but don't some other OV devices have a
separate Y channel (ie : 8 pins for Y and 8 pins for UV)?

Yes some are 16bit output, some of the earlier devices had an analogue
composite output as well as digital output, some were even analogue only.

It is a pity that devices such as OV519 are not available in small
quantities. I saw another device, a Sanyo LC82210 that would make image
capture a breeze. (I saw a simple block diagram here
http://www.alldatasheet.co.kr/datasheet-pdf/pdf_kor/SANYO/LC82210.html but
this part is not readily available.

OV do the 819 and 820 for similar process as do other companies.

 

[Paul Olley]

Hello Paul,

Thank you again for your reply.

That is the issue for a lot of people video is not teh same as graphics
cards
and those dealing with graphics think they have problems with resolutions.
Just wait till you have to deal with all the video formats!

Let me get the hang of YUV 4:2:2 first

I think the byte order is swapped on successive lines just like PAL
alternates
its colour phase, however the same edge determines which byte it is! So
the
starting phase may be different.

I think that explains the way I receive the image data.

Alternatively don't route the data through the PLD use TWO external
latches
to latch the data (16bit wide) on each phase, use nearly the same timing
in the
CPLD and write to TWO RAM chips at once from the outputs of the latches.

A good solution. Another I thought of was to add another SRAM, to extend
the available memory beyond 512k, and get the CPLD to enable each device as
appropriate.

Or two RAM chips or Field stores as you write in and read out
sequentially.
Field store chips are quite cheap.

I had a look for field store devices, all I could find at Farnell was around
£18 per device. Since I would like to do the JPEG encoding in a uC, (or
perhaps FPGA), and that will require (pseudo) random access, I will look at
an SDRAM solution. The added bonus would be that I could relatively easily
move up to a higher resolution sensor.

This is a chip scale package, primarily meant for chip on board hybrids
for
'throwaway' devices like toys, webcams and mobile phones. These devices
are
made in bulk as one batch maybe two batches, sold and forgotten about.

Do you mean that the manufacturer of the sensor does not intend to produce
it for some defined "lifetime"? (example : 3 years?) ie : They just care
to custom make it, for a mobile phone manufacurer say, for a specific model?
Interesting, I never knew that!

Alternatively instead of reinventing the wheel use one of the prebuilt
serial cameras that already have the guts in to do JPEG compression from
an Omnivision sensor.

I quite agree ; it is re-inventing the wheel, but I enjoy learning!

As it happens, I bought a camera from Intertec some time ago, which got me
interested in doing it "from scratch". I got the impression, perhaps
incorrectly, that comedia was the company that supplied Intertec?

Look at Field/Frame memories as these are cheap memories for video
recorders
and the like.

As above, but I will read more about these for interest.

I had some time this weekend to capture the colour information.

I proceeded as follows : set the registers (now that I can access them over
SCCB , to halve the PCLK frequency.

I only have 512kb, so by capturing 614400 bytes, I will wrap round 90112
overwriting the first 90112 bytes of the image.

My interest is currently in checking that I capture both UV as well as Y, so
I'm not concerned if I miss the first 90112 bytes.

I did this, and find that if I create an image with every other byte, (Y), I
get a good monochrome image. If I create the same image, this time offset
by 1 byte, I get an image, with the very little detail, suggesting that I am
capturing the UV information in those bytes.

Since SDRAM is inexpensive, and FPGAs of sufficient size are also quite
inexpensive (to create a controller), I will look at solving the problem
that way.

Before that however, I will work on decoding the YUV for display by an
applet.

Thank you again for your help.

Paul.

 

[Paul Carpenter]

On Monday, in article
<%[email protected]>

Hello Paul,

Thank you again for your reply.


Let me get the hang of YUV 4:2:2 first

Oh you mean you don't want to deal with Wide Screen Signalling and
zoom ratios, let alone interlaced to non-interlaced (progressive) and 50/60Hz
conversions, via a colour transcoding in SECAM... The youth of today!!!

I think that explains the way I receive the image data.


A good solution. Another I thought of was to add another SRAM, to extend
the available memory beyond 512k, and get the CPLD to enable each device as
appropriate.

The latches means you have less worries about timing delays through CPLD
paths, and have more stable timing without losing lots of I/O pins on the
CPLD. Get the CPLD to control the timing not the data path itself.

I had a look for field store devices, all I could find at Farnell was around
?18 per device. Since I would like to do the JPEG encoding in a uC, (or

Well if you want to grab frame and use other memory for jpeg afterwards,
Field memory manufacturers I would look at

Philips, On Semi

Oki Semi These two sold in UK by DT Electronics (Coventry)
Logic Devices

If you need the details for DT Electronics email me, as I have had dealings
with them and quite a few of their staff at various companies over many
years.

perhaps FPGA), and that will require (pseudo) random access, I will look at
an SDRAM solution. The added bonus would be that I could relatively easily
move up to a higher resolution sensor.

SDRAm is like PC memory a moving target that changes every 6 months (MAX)
and parts get obsoleted way too fast.

Do you mean that the manufacturer of the sensor does not intend to produce
it for some defined "lifetime"? (example : 3 years?) ie : They just care
to custom make it, for a mobile phone manufacurer say, for a specific model?
Interesting, I never knew that!

No the end product is a throwaway product. Unless you are building hundreds
if not thousands of these the costs of putting them onto PCBs can be expensive.

I quite agree ; it is re-inventing the wheel, but I enjoy learning!

Nothing wrong with learning, cameras is learning about many fields
at once.

As it happens, I bought a camera from Intertec some time ago, which got me
interested in doing it "from scratch". I got the impression, perhaps
incorrectly, that comedia was the company that supplied Intertec?

It could be, but I could not see similarities in the photos I saw on]
board layout.

I had some time this weekend to capture the colour information.

I proceeded as follows : set the registers (now that I can access them over
SCCB , to halve the PCLK frequency.

I only have 512kb, so by capturing 614400 bytes, I will wrap round 90112
overwriting the first 90112 bytes of the image.

If you have an address counter, or video line counter stop at overflow point
is easier.

My interest is currently in checking that I capture both UV as well as Y, so
I'm not concerned if I miss the first 90112 bytes.

I did this, and find that if I create an image with every other byte, (Y), I
get a good monochrome image. If I create the same image, this time offset
by 1 byte, I get an image, with the very little detail, suggesting that I am
capturing the UV information in those bytes.

Looks good for a start point.

Since SDRAM is inexpensive, and FPGAs of sufficient size are also quite
inexpensive (to create a controller), I will look at solving the problem
that way.

Before that however, I will work on decoding the YUV for display by an
applet.

Lots of nice matrix conversions. If you ordered the Video Demystified book
that will help you through Colour space conversion.

Thank you again for your help.

Too many years with video and computer systems..... Let alone the number of
different 8/16bit micros with video frame grabbers I have done.

 

[Paul Olley]

Hello Paul,

Oh you mean you don't want to deal with Wide Screen Signalling and
zoom ratios, let alone interlaced to non-interlaced (progressive) and
50/60Hz
conversions, via a colour transcoding in SECAM... The youth of today!!!

Maybe next week?

The latches means you have less worries about timing delays through CPLD
paths, and have more stable timing without losing lots of I/O pins on the
CPLD. Get the CPLD to control the timing not the data path itself.

The Altera 7032S CPLD currently just controls some 74HCT161 (count enable,
reset & preset) counters, and the 512k SRAM (ce, oe, we), with the data path
not being through the CPLD, instead the data flows from the OV7640 through a
74HCT244 and onwards to the SRAM. By manipulating the registers in the
OV7640, the data on the OV7640 can be tri-stated, meaning I could dispense
with the 74HCT244. However, I wanted the 244 for this first development
board.

Instead of two latches to capture the Y and either U or V data, why not just
capture the Y and U or V (directly into SRAM) by utilizing a clock,
synchronized to PCLK, but running 4 times faster than PCLK. That way you do
away with the need for the latches?

Well if you want to grab frame and use other memory for jpeg afterwards,
Field memory manufacturers I would look at

Philips, On Semi

Oki Semi These two sold in UK by DT Electronics (Coventry)
Logic Devices

If you need the details for DT Electronics email me, as I have had
dealings
with them and quite a few of their staff at various companies over many
years.

Many thanks for the suggestions. I will take a look at their datasheets.
Thank you also for the offer of details for DT Electronics.

No the end product is a throwaway product. Unless you are building
hundreds
if not thousands of these the costs of putting them onto PCBs can be
expensive.

Oh, I see. This interests me, I'll see what I can find on the web about how
they are assembled. Is it feasible to do it by hand (for a few prototypes?)

Nothing wrong with learning, cameras is learning about many fields
at once.

You are not wrong there!

If you have an address counter, or video line counter stop at overflow
point
is easier.

That would be a lot easier. Unfortunately since I use the "on time" of HREF
to gate the counter, I just don't have visibility of the count reached. I
could probably add a wire to the board, connecting A19 to the CPLD, and that
would indicate that the counter had clocked over to 524288 and that the SRAM
was full. I would need to think a bit more carefully about whether it was
possible to then prevent address 0 from being overwritten. I suspect it
might be possible.

Looks good for a start point.

The Y data looks fine to me ; I did some first tests with the U and V, and
seemed to get something that made sense with regard to the image I was
looking at.

I hope to spend more time on this soon.

Lots of nice matrix conversions. If you ordered the Video Demystified book
that will help you through Colour space conversion.

Ordered that today!

I expect, much like the YUV data rate, and the I2C, I will have a lot more
to learn

Too many years with video and computer systems..... Let alone the number
of
different 8/16bit micros with video frame grabbers I have done.

Experience is something that only comes with a lot of time invested!

Paul.

 

[Paul Olley]

Oh, I see. This interests me, I'll see what I can find on the web about

how they are assembled. Is it feasible to do it by hand (for a few
prototypes?)

Probably should have searched for this first before posting. I now follow
what this means. If it wasn't an image sensor I would have thought it
feasible to do it by hand, depending on the pin spacing. Through holes
could, with a lot of care, have a soldering iron applied from the "reverse"
side to solder the pad on the other side.

However, at 0.4mm between pads, (I have no idea what the OV parts pad
spacing are since I can't see a datasheet), I think it would be very
difficult indeed.

Perhaps best to stick to the SMD parts.

Paul.

 

[Paul Carpenter]

On Tuesday, in article
<[email protected]>
[email protected] "Paul Olley" wrote:

....

The Altera 7032S CPLD currently just controls some 74HCT161 (count enable,
reset & preset) counters, and the 512k SRAM (ce, oe, we), with the data path
not being through the CPLD, instead the data flows from the OV7640 through a
74HCT244 and onwards to the SRAM. By manipulating the registers in the
OV7640, the data on the OV7640 can be tri-stated, meaning I could dispense
with the 74HCT244. However, I wanted the 244 for this first development
board.

Instead of two latches to capture the Y and either U or V data, why not just
capture the Y and U or V (directly into SRAM) by utilizing a clock,
synchronized to PCLK, but running 4 times faster than PCLK. That way you do
away with the need for the latches?

A couple of advantages of two latches:-

1/ It becomes possible to write 16 bit wide at slower rate to SRAM

2/ Latches are easily tri-stated for dual porting of the RAM, so
the micro can read the SRAM. (possibly add an output from your
CPLD to drive an interupt for application acessing memory
whilst in frame grab).

3/ Having a 16bit wide SRAM interface can sometimes help with other aspects
of design interfacing for other sources of input. Make your grabbing etc
start with a 16 bit interface (Y:U/V) then whatever camera and other
source can be interfaced with that.

4/ 8 bit latches like 74[A]HC[T]574 are relatively cheap and have nice
in and out pinouts easier for board layout and pipelining (something
you will come across often in digital video). I do believe there is
also a package out with two or more latches in one SMD package.

Several years ago I was at a small video effects manufacturing company, with
small production runs and we would debate did we need 15 or 18 thousand of
these in any month...

.....

That would be a lot easier. Unfortunately since I use the "on time" of HREF
to gate the counter, I just don't have visibility of the count reached. I
could probably add a wire to the board, connecting A19 to the CPLD, and that
would indicate that the counter had clocked over to 524288 and that the SRAM
was full. I would need to think a bit more carefully about whether it was
possible to then prevent address 0 from being overwritten. I suspect it
might be possible.

I generally work with larger PLDs to contain the address counter, dual
porting of RAM between grabber and host as well as other timing functions.
This makes these sorts of controls easier, I have before now built video
test pattern generators in PLDs, one controlled two LCDs simultaneously
with each capable of being driven with DIFFERENt test patterns at 24bit
colour depth.

....

I expect, much like the YUV data rate, and the I2C, I will have a lot more
to learn

We all have more to learn, most of us do not realise it.

 

[Paul Olley]

Hello Paul,

A couple of advantages of two latches:-

1/ It becomes possible to write 16 bit wide at slower rate to SRAM

2/ Latches are easily tri-stated for dual porting of the RAM, so
the micro can read the SRAM. (possibly add an output from your
CPLD to drive an interupt for application acessing memory
whilst in frame grab).

3/ Having a 16bit wide SRAM interface can sometimes help with other
aspects
of design interfacing for other sources of input. Make your grabbing etc
start with a 16 bit interface (Y:U/V) then whatever camera and other
source can be interfaced with that.

4/ 8 bit latches like 74[A]HC[T]574 are relatively cheap and have nice
in and out pinouts easier for board layout and pipelining (something
you will come across often in digital video). I do believe there is
also a package out with two or more latches in one SMD package.

Some valid reasons. I think the most significant is the ability to use
other cameras.

I generally work with larger PLDs to contain the address counter, dual
porting of RAM between grabber and host as well as other timing functions.
This makes these sorts of controls easier, I have before now built video
test pattern generators in PLDs, one controlled two LCDs simultaneously
with each capable of being driven with DIFFERENt test patterns at 24bit
colour depth.

Having the address counter in the CPLD/FPGA would certainly make the board
layout somewhat simpler (and smaller.)

I'll try and put some time aside to look at what I currently capture, whilst
waiting for the book to arrive, and reading some of that to discover what I
have to do for decoding.

I noticed that areas of darkness are showing as white, I need to look at the
range of the bytes received for luminance, maybe I'm shifting something I
shouldn't be. I guess it could also be a hardware fault, best to check that
ALL bits, particular MSB are making a connection.

We all have more to learn, most of us do not realise it.

Paul.

 

[Paul Olley]

I'll try and put some time aside to look at what I currently capture,

whilst waiting for the book to arrive, and reading some of that to
discover what I have to do for decoding.

I have a very strange problem. Looking only at the Y component, I looked at
the number of times each luminance value appeared in an image. Certain
values of luminance never appeared.

Luminance values 128 to 204 inclusive were missing.

Looking at way the bits changed moving from 127 to 128 did not help me,
given that 204 through 255 were present.

My first suspect was port (F) of the PIC, but I checked to see that I had
disabled its peripherals to make it I/O only.

A JPEG, with compression set to zero, of the image in question is here :
http://homepage.ntlworld.com/paul.olley/index.html

I would expect the problem to lie with D7, however, that does not explain
why 205 through 255 are present.

Maybe I shouldn't spend too much time on this problem, which if it is
related to a hardware fault, will not matter since I need to change the
hardware architecture in any event to store the entire frame.

I checked another image, this time there were no occurances from 128 through
to 193 inclusive.

Any ideas?

Paul.

 

[Paul Carpenter]

On Thursday, in article
<eKhse.25303$%[email protected]>

I have a very strange problem. Looking only at the Y component, I looked at
the number of times each luminance value appeared in an image. Certain
values of luminance never appeared.

Remember that 0 and 254 are not allowed anyway (according to that preliminary
spec).

Luminance values 128 to 204 inclusive were missing.

Suggesting bit 6 is shorted possibly to bit 5

204 = CC hex
128 = 80 hex
--- --
76 4C

Looking at way the bits changed moving from 127 to 128 did not help me,
given that 204 through 255 were present.

Looking at a Histogram of the uploaded image gave gaps where strange groupings
of four pixels were zero counts repeatedly which is either how the picture
is converted into JPEG or other shorts/open data lines.

I would run a meter down adjacent lines into the PLD and to the RAM to find
shorts at each end and opens along the run.

My first suspect was port (F) of the PIC, but I checked to see that I had
disabled its peripherals to make it I/O only.

A JPEG, with compression set to zero, of the image in question is here :
http://homepage.ntlworld.com/paul.olley/index.html

I would expect the problem to lie with D7, however, that does not explain
why 205 through 255 are present.

I suspect D6 to D5 short or similar depending on layout.

Maybe I shouldn't spend too much time on this problem, which if it is
related to a hardware fault, will not matter since I need to change the
hardware architecture in any event to store the entire frame.

Best to make sure you get a good monochrome image to prove your timing
and capture is solid before commiting to the next design phase.

Also check that if you grab the first say 640x240 pixels of data you can get
good colour transforms (even if you change the image to 320x240).

Grabbing every 2nd pixel and every second line to give 320 x 240 resolution
may be a good interim fix for you. Leaving some part of the top end of your
SRAM available for working data area once grab has finished.

I checked another image, this time there were no occurances from 128 through
to 193 inclusive.

Any ideas?

See above..

You might find it easier to email me direct (replying to post will get to me)
or depending on where you are in the UK, you might want to spend some time
doing a beer and pizza session.Might have some equipment and ideas to
help you.

 

[Paul Carpenter]

On Thursday, in article
<o6gse.25074$%[email protected]>

Hello Paul,

A couple of advantages of two latches:-

1/ It becomes possible to write 16 bit wide at slower rate to SRAM

2/ Latches are easily tri-stated for dual porting of the RAM, so
the micro can read the SRAM. (possibly add an output from your
CPLD to drive an interupt for application acessing memory
whilst in frame grab).

3/ Having a 16bit wide SRAM interface can sometimes help with other
aspects
of design interfacing for other sources of input. Make your grabbing etc
start with a 16 bit interface (Y:U/V) then whatever camera and other
source can be interfaced with that.

4/ 8 bit latches like 74[A]HC[T]574 are relatively cheap and have nice
in and out pinouts easier for board layout and pipelining (something
you will come across often in digital video). I do believe there is
also a package out with two or more latches in one SMD package.

Some valid reasons. I think the most significant is the ability to use
other cameras.

Especially if the other types of interface (e.g. multimedia chip with 16 bit
interface) is connected to the tri-state side of the latches.

Having the address counter in the CPLD/FPGA would certainly make the board
layout somewhat simpler (and smaller.)

The part I described above fitted into about 3/4 of Altera 7128 device 100pin
TQFP plus 6 x 8 bit buffers to drive the lines to LCDs.

I'll try and put some time aside to look at what I currently capture, whilst
waiting for the book to arrive, and reading some of that to discover what I
have to do for decoding.

I noticed that areas of darkness are showing as white, I need to look at the
range of the bytes received for luminance, maybe I'm shifting something I
shouldn't be. I guess it could also be a hardware fault, best to check that
ALL bits, particular MSB are making a connection.

I think you have other bits shorted causing the problems, see other post.

Paul.

Contact me in email if you wish to discuss methodologies etc..

 

[Paul Olley]

Hello Paul,

Suggesting bit 6 is shorted possibly to bit 5

204 = CC hex
128 = 80 hex
--- --
76 4C


Looking at a Histogram of the uploaded image gave gaps where strange
groupings
of four pixels were zero counts repeatedly which is either how the picture
is converted into JPEG or other shorts/open data lines.

I started by checking for shorts with a multimeter. I also checked end to
end connections. Lifted the lens off the image sensor and checked the
connections there too.

Not finding anything, I proceeded as follows :
Use the PIC to write 255 bytes into SRAM, write data of 0 into address zero
and incrementing the data value at each step so that finally at address 255
the data value was 255.
Then, reset the address counter, and read the results back. Perhaps I
expected this, but the result was that on each read, the correct value was
received.

This suggests that the problem may lie before the first latch which passes
the data through from the OV to the SRAM or perhaps I have a timing issue.
I'll look further at both possibilities.

Best to make sure you get a good monochrome image to prove your timing
and capture is solid before commiting to the next design phase.

Particularly if it is a timing issue!

Also check that if you grab the first say 640x240 pixels of data you can
get
good colour transforms (even if you change the image to 320x240).

Grabbing every 2nd pixel and every second line to give 320 x 240
resolution
may be a good interim fix for you. Leaving some part of the top end of
your
SRAM available for working data area once grab has finished.

Not a bad idea ; grabbing every other pixel and skipping every other line
would be a good way to get a valid image.

You might find it easier to email me direct (replying to post will get to
me)
or depending on where you are in the UK, you might want to spend some time
doing a beer and pizza session.Might have some equipment and ideas to
help you.

Will email you directly.

Paul.

 

[Paul Olley]

A JPEG, with compression set to zero, of the image in question is here :

http://homepage.ntlworld.com/paul.olley/index.html

Since the above webpage is still getting a significant number of hits 8
months later, to make the URL "permanent", I've copied the HTML to the
following URL :
http://www.olley.com/ov7640/index.html

The original will disappear in due course.

Paul.