Maker Pro
Maker Pro

Re: how to read AIS data from encapsulated NMEA VDO sentence

T

Terry Spragg

Pascal said:
Disassembling message protocols is tedious by hand. Automated
equipment does it with logic circuitry, which accomplishes tedious
tasks easily and knows the secret code.

We humans must parse manually.

The data is actually send as a bitstream and the protocol for every
device is proprietary.

The data presented above is actually encoded as hexadecimal or
ascii encode binary characters, so it actually looks more like
111101100101101010101010101011011001100110101010101010111010101...,
etc. These bits are sent as analogue levels, data encoded in one
long serial string. 100 character might be 800 bits, or 1600 bits,
in number base 10 notation. It might use another numerical base,
likely octal or hexadecimal.

The ports have their own protocols, and take each byte of 7 or 8
bits (as one example, I am not certain what this one actually uses,
perhaps 16 bit bytes) as a character. Each group of bits represents
one character in the string given above. The alphabet used by the
system may have many more than the 26 in our human alphabet, and
some of those characters are specific to the device in use. The
spacing and syntax is combined with error correcting parity (even -
odd) codes and checksum numbers which must also be decoded in logic
if you are writing an interface. Each receiving device must be up to
date with whatever protocol is used by all of the transmitters if it
is not to become discombobulated and confused, losing track of
character boundaries, etc, and those protocols seem to change with
each new generation of devices.

Different portions of the data string may use different schemes, ie,
the first 6 bits might be all ones, as a preamble and sync. The
next 4 bits might be a code to identify the remainder as a specific
type of statement, ie, sender I.D. Sometimes the info is bitsliced,
so that each bit represents a switch setting in the receiving parser
to cause it to understand that it is radar, weather, whatever.

I once independantly decoded the data sent by telco in it's caller
id string, sent between the ring tones to your phone. There are
padding bits, a preamble, and then each character is sent twice. It
was painful for a while, as I had to scope out the data without
benefit of knowing the protocol, even as simple as it was, using a
capture buffer to read each character out in ascii values, a sofware
protocol analyzer I wrote myself, which spit out gibberish ascii
codes until I learned to read it. If it had not been straight ascii,
normal in telephone modems, I would still be propping up my eyelids
with toothpicks.

You seem to have a protocol guide or data map, which you must now
come to know imtimately. Reverse engineering software data is fun
for some masochists, for a while.

400 page logic diagrams is where I cut my teeth, back in the 60's.
It's one way to burn out your brain so far as computers is
concerned. I got fed up with them, and retired years ago. Wouldn't
touch it, now, without cash up front and a clearly defined mission
statement engraved in titanium.

Good luck.

Terry K -SofDevCo-

[email protected]
 
T

tcdang714

Terry, you're making this sound alot more complicated than it really
is.

The 6-bit encoded part of a VDM sentence is simply a bitstream, packed
using a special code. Each character represents 6 bits of data from
0x00 ot 0x3F. It starts with the message ID (6 bits), repeat (2 bits),
MMSI (30 bits), etc. This is all defined in m.1371 and 61993-2, eve
if
it is a bit confusing at first.

The first 6 bits being a character and the message ID make it easy to
tell that the example string above is a message 1.


Hi guys,

I'm actually working on a similar project right now except I have to d
the ENCODE part of the VDM message. At first I thought every characte
represents 6-bit but that is not true. As you can see in the messag
there are lowered case alphabet characters (i.e. 'e') and the asci
value for 'e' is more than 6-bits. I'm a little stuck on my projec
right now too, can someone give me more info. Perhaps an example in
or C++. Thanks
 
P

Paul

tcdang714 said:
Hi guys,

I'm actually working on a similar project right now except I have to do
the ENCODE part of the VDM message. At first I thought every character
represents 6-bit but that is not true. As you can see in the message
there are lowered case alphabet characters (i.e. 'e') and the ascii
value for 'e' is more than 6-bits. I'm a little stuck on my project
right now too, can someone give me more info. Perhaps an example in C
or C++. Thanks.

I realize that you've asked about *encoding*, but here's an example of a
decoder written in visual basic, from a program I wrote for my pocket PC.
Perhaps you will be able to reverse-engineer something from this.
------------------
Function AsciiToHex6(AsciiIn)
AsciiToHex6 = A2H6(Asc(AsciiIn))
End Function

Dim A2H6(128)
Sub InitA2H6
Dim i, h
For i = 0 To 127
h = i
If i < 48 Then
h = -1
ElseIf i < 88 Then
h = h - 48
ElseIf i > 119 Then
h = -1
ElseIf i < 96 Then
h = -1
Else h = h - 56
End If
A2H6(i) = h
Next
End Sub

InitA2H6
------------------
The array A2H6() contains the hex6 value of the ascii input character, using
the character as an index. I put "-1" in the illegal positions, and ought
to do a little more defensive error-checking (but I probably won't ever get
around to it). The function AsciiToHex6(AsciiIn) just looks up the code and
returns it.

The IEC documents that describe the messages and coding aren't free, but
while I was googling around I did find this one:
http://www.gicoms.go.kr/knowledge/d...LE\updown\IEC standard 61993(class A AIS).pdf

This contains a decent description of the 6-bit ascii encoding, as well as
the message structure. Note that the AIS data elements are not necessarily
six-bits long, so there will be an arbitrary alignment of the various
parameters across the 6-bit-ascii character stream. Also, some of the AIS
messages are longer than allowed in the NMEA sentence, so the messages have
to be split into multiple NMEA sentence. I can post my code for decoding
the arbitrary data if you think it would be instructive.

Be aware that some of the earlier discussions of 6-bit ascii encoding for
AIS show a different encoding (don't ask me how long I struggled with that
issue!). The one in my code sample, and the document above seem to be the
encoding actually used.

Good Luck,
Paul
 
P

Paul

Actually, I re-read what I posted and it didn't really make sense.

1. Start with the array of ascii characters
2. Convert to the 6 bit binary value.
3. Convert this byte array to a 6 bit bitstream.
4. Then just pull out the bits yuo want using the various bit
operators. I tried to use a bit-field, but it didn't seem to work in
Windows.

As I said, it all works until I get to character arrays (call sign etc)

Here is a portion of the code i wrote to handle message type 5 (I'm only
handling the "Ship Static and Voyage Related Data
" variant for now). My comments follow the code. Pay attention to my
function H6StrToAsc, and how I manipulate the ascii.

-----------------
Case "5"
MMSI = H6StrToInt(AISString, 9, 38)
ndx = FindAISRecord(MMSI)
If ndx < 0 Then Exit Sub 'not found and no room for new record --
discard report! (need to announce)

temp = H6StrToInt(AISString, 39, 40)
Select Case temp
Case 0 'Ship Static and Voyage Related Data
IMO = H6StrToInt(AISString, 41, 70)
ShipTypeStr = ShipType(H6StrToInt(AISString, 233, 240))
EtaMin = H6StrToInt(AISString, 289, 294)
EtaHour= H6StrToInt(AISString, 284, 288)
EtaDay = H6StrToInt(AISString, 279, 283)
EtaMonth= H6StrToInt(AISString, 275, 278)
Draft = H6StrToInt(AISString, 295, 302) / 10
Callsign = H6StrToAsc(AISString, 7, 71)
ShipName = H6StrToAsc(AISString, 20, 113)
Destination = H6StrToAsc(AISString, 20, 303)

Call CleanAt(Callsign) 'need to strip off any trailing '@'
characters some ships have
Call CleanAt(ShipName)
Call CleanAt(Destination)

AISArray(ndx, 0) = MMSI
AISArray(ndx, 15) = ShipName
AISArray(ndx, 16) = Callsign
AISArray(ndx, 17) = Destination
AISArray(ndx, 8) = CStr(EtaMonth) & ":" & CStr(EtaDay) & ":"
& CStr(EtaHour) & ":" & CStr(EtaMin)
AISArray(ndx, 11) = 0 '"Length val"
AISArray(ndx, 13) = Draft
AISArray(ndx, 14) = ShipTypeStr
AISArray(ndx, 18) = vbBLACK 'Plot Color

If AISArray(ndx, 29) = 1 Then Call DoAISLog(ndx) 'first time
a complete record
AISArray(ndx, 29) = AISArray(ndx, 29) Or 2
Call UpdateAISDisplayListElement(ndx)

Case 1 'Extended Ship Static and Voyage Related Data
ErrorReport("Extended Ship Static And Voyage Related Data: "
& AISString)

Case 2 'Aids to Navigation Data
ErrorReport("Aids to Navigation Data: " & AISString)

Case 3 'Regional Ship Static and Voyage Related Data
ErrorReport("Regional Ship Static and Voyage Related Data: "
& AISString)

End Select
-----------------
Sub CleanAt(str) 'used to strip off any trailing '@' characters some ships
leave in their strings
str = Left(str, InStr(str, "@")-1)
End Sub

-----------------
Function H6StrToAsc(str, characters, MSbit)
Dim tempstr, tempchar
Dim i
For i = 1 To characters
tempchar = H6StrToInt(str, MSbit, MSbit + 5)
If tempchar < 32 Then tempchar = tempchar + 64
tempstr = tempstr & Chr(Tempchar)
MSbit = MSbit + 7
Next
H6StrToAsc = tempstr
End Function
-----------------

I decided to not convert the raw NMEA ascii string into an intermediate
form, but rather suck the bits out of the string as I need them. My
function H6StrToInt(AISString, starting_bit_position, ending_bit_position)
returns a the value of a particular bit-field.

The function H6StrToAsc(str, characters, MSbit) reads the encoded characters
from the NMEA ascii string, converting them from bits to numbers to
characters. I test the characters (char < 32) and map those that are
between 0 and 31 (decimal) to the ascii characters between 64 and 95. I
don't remember exactly how I arrived at this manipulation, but I am pretty
sure I had to just look at the data and figure it out. There is a nice
application on the web that decodes some of the simple AIS strings, and I
probably used that to test my own decoding. Here is the URL:
http://rl.se/aivdm

Once you've got the decoder working, you can use the ITU "Search for Ship
Particulars" website to find out more about a specific ship, given it's
callsign or MMSI or name:
http://www.itu.int/cgi-bin/htsh/mars/ship_search.sh

Back to my code -- I just re-learned BASIC to do this application, after not
using it for over 30 years. I'm sure that my style is horrible, but I doubt
that I will attempt to improve it.

Oh yeah, in my code AISArray is where I keep the AIS records. I don't
remember why I used the intermediate variables instead of storing the data
directly in the array -- this is probably just how the code evolved and I
didn't want to go back and clean it up.

Good Luck!
-Paul
 
P

Paul

I'll assume Paul's code is right :) He's skipping the intermediate step
which may be where the confusion lies. There are several layers of
things happening here.

0. Bits over the air: 9600bps GMSK data.
1. Demodulated bitstream, eg. 168 bits for a message 1.
2. Actual message content like Message ID, MMSI, Ship Name.
3. Serial output of bitstream using 6-bit encoding.

The data you see in the VDM sentence is a 6-bit encoded representation
of the raw bitstream, first bit on the left, last bit on the right (as
God intended things!).

To get the ship name you need to either skip ahead like Paul did, or
convert the 6-bit back to its bitstream. Then you grab the bits in the
name position and apply the m.1371 ASCII encoding rules to those bits.
BUT! That conversion is DIFFERENT from the 6-bit encode/decode. That is
probably what was tripping you up.

.cp

I hereby give you a full money-back guarantee that my code is correct. At
least it seems to usually kind of work. The only trouble I've seen is that
I get a spurious (or incorrectly encoded?) rate-of-turn value sometimes,
positive or negative full-scale, if I recall correctly. I haven't posted
that code, though.

As for the intermediate step of converting to a serial bitstream, there is
certainly nothing wrong with doing so, and it may indeed simplify some
operations. I found that with the lack of easy bit-field operators in
Visual Basic, it was just as simple to use the VDM message character string
directly. All the data is there, and I found no good reason to do an
additional conversion. If I were dong this in c, I probably would have gone
to the bitstream. And yes, the ASCII conversion is different than the 6-bit
encode/decode, which is why I directed the original poster to the H6StrToAsc
function in my code.

I was about to ask for help with my ROT problem, and in researching my
sources I just found a later spec than I had been using which shows that
+/-127 values have been taken over to indicate a heading-derived ROT, rather
than a turn-indicator-derived ROT (which uses the values between -126 and
+126). Problem solved! Now, I guarantee double your money back!

Regards,
Paul
 
Top