370Z CAN Bus Proprietary PID's
 

Im not sure if this is the right place for this...

I wanted to start a thread that could serve as a central location for complete CAN Bus decoding. There is a great deal potential in being able to understand the non-stnadard PID's flying around the Z CAN bus.

I have some CAN hardware on order. My plan is to log the bus information during normal driving. Who's up for some data analysis!? :ugh2:

My thought is to attempt to analyze the logged data to see what PID data changes and when to try to figure out what is going on and create more complete documentation.

Current sources:
[370Z]
https://github.com/Knio/carhack/blob...issan.markdown
Nissan 370Z CAN Hacking | Stray Bytes
https://docs.google.com/spreadsheet/...=sharing#gid=0
http://www.the370z.com/audio-video/6...s-project.html

[Other Nissan]
http://www.mynissanleaf.com/viewtopic.php?f=44&t=4131

If anyone has more information or sources, please contribute!

Is there a stereo on this bus? lol

Straybytes.com is my blog. I haven't worked on that in a while because I've been distracted by other projects. I can help if you have any questions though.

I am currently running this app on my phone. Looks like they have mapped out a bunch of the Nissan only PIDs...

https://play.google.com/store/apps/d...eeo.niadvanced

My Nissan Leaf Forum • View topic - LEAF CANbus decoding. (Open discussion)

They are trying to decode the Nissan Leaf CAN bus, but there are many similarities.

I wonder if its possible to see the CAN data to understand what the bytes for each PID mean??

Sweet usable data ...
 

Just got my CAN bus interface hardware. I wrote a simple program that reads the data on the CAN and prints the PID and data in HEX to a console.

https://dl.dropboxusercontent.com/u/...0Z_CAN_log.PNG

Let the reverse engineering continue!

I have an ELM-based CAN-to-USB adapter. Would it be possible to get a copy of your program?

I don't believe my code will help you in your case. My code is C++ firmware code that gets loaded to an Arduino and communicates directly to a MCP2515 CAN controller chip via the SPI protocol. That data is then translated to UART to communicate to the PC to see the results.
https://cdn.sparkfun.com//assets/par...0/10039-01.jpg

I just used Hercules to view the COM port.

RS232 terminal, serial port terminal, udp terminal, tcp/ip client server - Hercules

The hardware you have should do this already. If you use Hercules, you can log data just as I have.

The advantage of having such a low level interface is that once I start understanding what the data is, I can program my Arduino to start doing stuff with it!

I am writing a program now that will allow me to view all the data on the CAN simultaneously by PID. That way, I can start messing with stuff in the car and see what data changes and how.

Ah. OK.

The hardware/software I have found work great for monitoring params if you know the PID, but I haven't found a free program that will do bus sniffing.


Please keep us posted on your progress.

Update
 

I wrote a test tool for the incoming CAN data. The program reads in the CAN stream from the arduino and displays unique ID's and their data length. When the ID repeats, the line containing that ID gets updated. This allows me to view a static arrangement of all the PID's and watch how their data changes in real time.

Output format:
XXX: XX XX XX XX XX XX XX XX
PID: D0 D1 D2 D3 D4 D5 D6 D7

https://dl.dropboxusercontent.com/u/...IDs_Python.PNG

Introducing the CANBus Triple! - Mazdaspeed Forums

Back when I had my Mazdaspeed3 This guy was messing around with doing a CANBus interface unit for the car. He seems to have some pretty good posts outlining the process he used to get it working. I know it's not the same car, nor do I have experience with it, but maybe something helpful for you.

Nice! I've been thinking about the same project in which I want to output to a composite video input of my head unit with the Arduino. I want to refresh the entire screen instead of printing it out in log format. The only issue I have is the resolution, so I thought about using a Raspberry Pi and write a Java app to make it look pretty on the screen with a touch friendly UI. Among other features, real-time plotting like an oscilloscope, bar charts, and the ability to view peak and hold.

All I need to do is query the data in real-time (similar to the console output) given the list of PIDs, and then parse the raw data.

This is something I'm already doing at work in the .NET world. Although my other option is to get a hold of Mono for Visual Studio so I can quickly code stuff in C#, but then it'll be an Android app.

BTW... what sort of response do you get if the PID doesn't exist?

He's just listening in on the CAN bus and printing it to a screen. You don't have to send any commands unless you want to start querying the OBD2 type information. The information he has printed is actually used by the car to function.

radensb, when you start logging we should decide on a standard format so we can exchange logs and input them into whatever programs we write.

Here's an example I did early on using excel, but my VB.net program is coming together slowly which automates the process of comparing and graphing information to figure out what they mean.

I start by backing out of my driveway and then I drove around my neighborhood a bit (reverse = -1, neutral = 0, first = 1, etc.).

http://www.the370z.com/attachment.ph...4&d=1400014556

Wah, so many skillfull programmer here.

Gents after a long journey trying to understand why JDM car unable to read the bloody data on torque i can safely say that now with the new Nissan adv plugin on torque we (jdm) ecu can actually read from it, but limited to the custom pid by the plugin only. Im trying to figure out how to read the standard mode 1 pid but so far unsuccessfull.

Can someone teach me how to put the custom pid into torque, maybe it can force torque to read it that way. Maybe the developer just bypass the polling routine for standard pid since requesting for 0000 turning null which mean the ecu did not support any pid.

I would like to suggest that the moderator move this thread to tuning section since it is more related to ecu. TS, if you dont mind, i made a post at torque bhp forum to this thread and your spreadsheet at

http://torque-bhp.com/forums/?wpforu...topic&t=4875.0



Car is 370z 2009 7AT JDM

Latest spreadsheet
 

I did a little updating and added new information, plus I put the bits in the proper order. My spreadsheet is now hosted on Dropbox to make it easier for me to keep it synced.

https://www.dropbox.com/s/cfpzb2mpj4...70Z%20CAN.xlsx

So the google spreadsheet listed in your first post has been deleted.

This makes it so much harder to hold off on adding CAN-bus communications to my homemade datalogger. So much useful info I can get from the ecu, especially steering angle, brake position, and throttle position.

awesome work! I have been side tracked with other projects but this will save me a bunch of time when i get back to my carputer. I'm using a raspberry pi, bluetooth CAN (STN1110), 10-bit 8 channel ADC and a bunch of 5v analog sensors (boost/vac, oil pressure, fuel pressure etc).

http://i.imgur.com/mCe4un2.jpg

That's basically what my ultimate goal is. Maybe add a touch screen display into the cubby above the stereo. Or just show information on my Kenwood stereo display if the resolution is good enough.

yeah mine is going in a modified cubby, 7" pi compatible touchscreens with transflective panels sell for about $120 on ebay. they work great!

A somewhat silly use for this also occured to me since we can see the status of various buttons and switches in the car by monitoring the CAN bus. Here are 2 ideas:

This can be used to enter a 'code' to activate something or to identify the driver to the carputer. A code such as tapping the brake 5 times quickly.

Or we can repurpose buttons for something else. I found the bit that shows whether the key is inserted into the slot beneath the VDC button (I'll add it to my spreadsheet soon). It's just a button that can be pushed if you insert your finger all the way back. Just monitoring this bit can activate something and it makes for a very stealthy switch.

Not silly at all.

The theft-deterrent idea should be pretty easy to implement with a relay in the push button or starter circuits.

You should be able to come up with a "valet mode", using some switch (or combination) to limit RPM. I'm not sure how one would limit RPM, but you seem to have enough access via CAN to do it, if it's possible.*

The possibilities are endless!



One thing that I've wanted on my 7AT is to have the current gear displayed on the dash in automatic mode, as it does when in manual mode. Not sure how the dash handles it, so it may not be possible.

*Edit: It sounds to me that what you are working on is basically a "dumb" tuner. If you know the right address to poke data into, you should be able to do everything the tuning software does, albeit without the benefit of the reverse-engineering done to identify all the "good" addresses.

The ECM sends the current gear to the dash repeatedly (every 55ms on my manual Z). So the problem is that even though you could send out messages disguised to look like the message from the ECU to trick it into displaying the current gear, it would immediately be replaced with the next message from the actual ECU.

Wouldn't that message be the next gear you are in, or do you mean it would keep cycling between "D" and a numeric gear value?

I believe SouthArk370Z wants to be able to replaced the 'D' with the actual numeric gear value. But I think you would be in a tug-of-war with the ECU and the display might be unreadable because it would bounce back and forth between D and the gear number. Or it might cause errors.

One way to find out, lol.

I've received a couple PMs asking what tools I'm using to sniff the CAN bus. Below is a pic of the VB.net app I wrote to display the info. I took a cue from radensb's screenshots for layout.

The left panel shows the CAN messages. Once the Lock Values button is pushed, all of the current values are saved and if any of them change they are highlighted in red. This makes it easy to see changes.

If you click on a byte, it will show the individual bits in the Show Bits section. Then you can click on a single bit, which will lock the green block to that bit. It will show green for 1 and red for 0. This makes it easy to see changes from far away when messing with the car.

I'm going to work on this some more over the next couple weeks before I let anybody else use it. I still need to add OBD PID polling so that info can be logged too. Just thought I'd share. It only works with a CANUSB adapter.

http://www.the370z.com/attachment.ph...1&d=1403177361

C, C#, Java, etc., devs UNITE!
 

Here are some ideas off the top of my head:

• Use an iDictionary to create a list of PIDs <int,class_object>, in which "class_object" defines fields to store the raw value and other properties of the PID.
• Use the iDictionary list of PID objects to populate a checked Listbox control. Each ListBox item stores the ID, which can be used to look it up in the iDictionary. Add options to "Select All" and "Select None." I love LINQ since you can write 3 lines of code to "query" the List of objects without the need to write recursive search.
• Add a search box that filters the list as you type. An "X" button to clear the textbox, which unfilters the list.
• Also, you can add a toolstrip with the standard button layout. The save button can be used to export the data to a comma or tab delimited file.
• Create a database which defines the PIDs and properties. These properties are used to decode the raw values. You'll need to research this part (e.g., data size, data type (int, double, string), resolution, bit range, etc.). You an put it in an MDB, SQLite, or even XML. On startup, you can read the contents to populate the iDictionary, which gets loaded into the ListBox. What you can do is create a table that defines the raw PIDs. The next table can break down the raw PID data into bit fields. For example the PID table can list IDs 000,001,100,200,etc. The next table can define 100-1, 001-1, etc. If the PID contains multiple fields of values in a single byte stream, you can define bit fields from there. You can create a column for the bit field range LSB and MSB. If it is equal, then you're defining a single bit. Then you can define the resolution, in which you do the basic math for the bit field and multiply the resolution to get the final result. Another column can be defined for units as well. If you need me to help you brainstorm the relationships, feel free to contact me.
• Add a control to let users adjust the polling interval.
• Add "playback" controls, which can scroll through the CSV log file. Don't forget to time stamp each row of data, so it can be read back.
• Add scanning capability in which one can create a logic (e.g., when value ==, !=, <, >, etc., some threshold).
• For plotting, I've had good luck with Zedgraph. It does have some issues that require some fiddling with code to workaround them.

The VS project I have at work is a tablet friendly real-time viewer which manipulates HTML displays, so I can flip images to simulate control state, update text values, and even set text colors. One can apply logic so perhaps the text color can indicate out of range values. HTML lets people create screens beyond looking like tables. I can probably port the entire project for ODB. Below is a program I wrote at work that reads XML files which can export the list as comma or tab separated with filtering capability.
http://www.the370z.com/attachment.ph...1&d=1403185575

I'll try to find time to come up with a tablet-friendly .NET application. It's based off the data viewing tool I wrote at work. Design plan:

• Tablet-friendly interface (iPad-like UI)
• Module for ODB communication. It can send commands and retrieve data. That's something you guys can help me with.
• Module for real-time module with recording capability. The sampling rate is user selectable. Once real-time is disabled, the module switches to playback mode. This will allow users to step and scroll through the recorded snapshot. Users will also be able to specify the buffer size by number of samples. Users will also be able to save and load this recorded file for playback.
• Viewing tools: HTML display, editor, Raw PID viewer, Event Search, Event Viewer (scans for out of range values), and an 8-channel plotter. The tools can support real-time and data playback modes. However, Event Search can only scan through the recording buffer.
• ODB Tools: Come up with general tools like diagnostics and commands. This might be database driven.
• For expandability this .NET program will database driven. Depending on how granular we want this to be, we can separate these definitions by make and model, or even down to the model year. A single database keeps track of this list, so it knows which set of definitions to get.

[EDIT] Already started. I copied the project and started deleting non-essential stuff.

I did a little experimenting with DotNetBar gauge controls. It isn't pretty, but it's a proof of concept and they all update in real-time.

http://www.the370z.com/attachment.ph...1&d=1403991394

I may have hit the mother load.

I found a way to access Nissan's proprietary PIDs. They aren't found using the standard mode $01 described here and they are 2 bytes long, not just 1. They are accessed using mode $22, but the ECU needs to be placed in a special diagnostic mode (I don't think this will impact driving at all since the dealership needs to run the car while testing it too, but I still need to try it out). I describe how I figured this all out on my blog here.

All in all, I found 254 custom PIDs that are completely undocumented so it's going to take quite a bit of work to decipher them all. Please help if you can. Hopefully this will give us access to a lot more sensors and data. I'll be updating my progress on here and on my spreadsheet. They're on the "Extended PIDs" tab.

Here's what I got with the key in the ON position, but the car not started. It goes PID, # bytes, then the data bytes.

1100 4 E4 41 9A 63
1101 1 61
1102 1 00
1103 1 96
1104 1 50
1105 1 00
1106 1 69
1107 1 00
1108 1 00
1109 1 00
110A 1 00
110C 1 C8
110D 1 5C
1110 1 00
1111 1 50
1112 1 00
1113 1 00
1114 1 23
1115 1 C4
1116 1 C4
1117 1 00
1118 1 00
1119 1 00
111A 1 1C
111B 1 1C
111C 1 00
111E 1 00
111F 1 5F
1120 4 3C 4C 01 81
1121 1 00
1122 1 00
1123 1 64
1124 1 64
1125 1 64
1126 1 64
1129 1 C4
112A 1 D1
112B 1 80
112D 1 00
112E 1 00
1130 1 00
1131 1 00
1132 1 00
1133 1 00
1134 1 40
1135 1 80
1136 1 00
1137 1 00
1138 1 00
1139 1 00
113A 1 80
113B 1 00
113C 1 00
113D 1 00
1140 4 07 06 00 C1
1143 1 00
1144 1 00
1145 1 01
1146 1 00
1147 1 58
1148 1 70
1149 1 00
114A 1 00
114C 1 00
114D 1 00
114E 1 00
114F 1 00
1150 1 00
1159 1 00
115A 1 00
1160 4 00 00 01 CD
1162 1 00
1163 1 80
1164 1 00
1165 1 00
1166 1 80
1174 1 00
1178 1 00
1179 1 C2
117A 1 63
117B 1 00
117D 1 15
117E 1 15
117F 1 00
1180 4 21 E0 00 00
1182 1 FF
1183 1 00
1188 1 00
1189 1 00
118A 1 00
118B 1 00
118C 1 00
1200 4 BF AF 10 71
1201 2 00 00
1203 2 00 00
1204 2 00 42
1205 2 00 42
1206 2 04 CC
1207 2 04 CC
1208 2 AB 07
1209 2 00 BF
120B 2 00 00
120C 2 00 00
120D 2 00 92
120E 2 00 8E
120F 2 00 7B
1210 2 00 7B
1214 2 01 40
1218 2 02 72
1219 2 00 00
121A 2 00 00
121B 2 15 76
121C 2 21 F7
1220 4 EC 60 E0 01
1221 2 00 AE
1222 2 00 D3
1223 2 00 EB
1225 2 01 B9
1226 2 01 B9
122A 2 02 15
122B 2 00 00
122C 2 00 00
122D 2 00 00
122E 2 00 00
122F 2 00 00
1230 2 00 00
1231 2 56 33
1232 2 45 56
1233 2 30 41
1234 2 01 18
1235 2 00 00
1236 2 00 00
1237 2 00 EF
1238 2 00 C2
1239 2 00 E4
123A 2 00 19
123B 2 00 B1
123C 2 01 44
123D 2 00 80
123E 2 00 00
123F 2 00 3B
1240 4 03 FD 00 6D
1241 2 00 CA
1243 2 00 75
1244 2 01 B1
1245 2 01 B3
1246 2 00 00
1247 2 00 03
1248 2 B6 3A
1249 2 FF D7
124A 2 FF D3
124B 2 00 79
124C 2 00 7B
124D 2 80 00
124E 2 80 00
124F 2 80 00
1250 2 00 03
125A 2 00 83
125B 2 00 82
125C 2 02 0E
125D 2 01 71
125E 2 01 BE
125F 2 FF FF
1260 4 03 80 C0 00
1267 2 00 E0
1268 2 01 15
1269 2 00 00
126B 2 01 BB
126C 2 01 BB
126D 2 01 DD
126E 2 01 EE
126F 2 01 BB
1270 2 01 BB
1271 2 01 65
1272 2 02 0F
1273 2 00 7B
1274 2 00 97
1275 2 01 BB
1276 2 01 BB
1300 4 7B FC 04 00
1301 2 00 00
1302 2 01 3F
1303 2 C0 30
1304 2 80 90
1305 2 68 44
1306 2 00 00
1307 2 00 42
1308 2 01 08
1309 2 10 02
130A 2 10 10
130B 2 00 03
130C 2 E5 E5
130D 2 10 3F
130E 2 00 7F
130F 2 DB FF
1310 2 00 03
1311 2 00 00
1313 2 00 00
1315 2 00 00
1316 2 11 1F
1317 2 00 00
1500 4 5B DB 62 00
1502 2 00 09
1503 2 00 00
1504 2 00 FF
1505 2 00 09
1506 2 00 00
1507 2 00 FF
1508 2 00 09
1509 2 00 C0
150A 2 00 3C
150B 2 00 00
150C 2 00 C0
150D 2 00 3C
150E 2 00 00
150F 2 00 C0
1510 2 00 3C
1511 2 00 00
1512 2 00 C0
1513 2 00 0C
1514 2 00 00
1515 2 00 00
1516 2 00 00
1517 2 02 02

That's pretty sweet. Unfortunately, VS failed on me and I nearly lost half of my project at work!

I created a new blank solution so I can import my existing project. I started deleting stuff and making changes. Then I needed to go back to my work project and saw a bunch of errors. It turns out VS didn't "clone" my project and copy it to the solution. :shakes head:

So... I'm going to make a duplicate copy next time so I don't lose my stuff. I'm in the middle of releasing it, so I'll get back to it in a few weeks.

Does anyone have friends at a garage that can hook up a Consult to a 370Z, make it view as much data as possible, and then send me the log files from it? This is going to be almost impossible to decode without cheating like that.

I do know that the 1100, 1120, 1140, etc. PIDs work the same way as the mode 01: 00, 20, 40, etc. PIDs. They signify which of the following bits are valid. All of the rest is actual data. It looks like 1100 & 1200 series PIDs are mainly sensors (speeds, temps, pressures, etc.) The 1300 & 1500 series PIDs looks like statuses of various things in the car. So once I figure out what those represent it's possible that I can manipulate them directly (unlock doors, VDC off, headlights, etc.) but I still need more time to play around.

Also, I drove around and logged a bunch of information and confirmed that you can drive perfectly fine while the ECU is in the diagnostic mode required to get the proprietary PIDs.

Nice find. I'll see if i can get torque + elm 327 to go into diagnostic mode like you did by creating a custom pid and put the initialization string on diagnostic command. If i can at least get the standard pid then it will be good because my car is jdm and it is not obd2 compliance. Only manage to get 11 pid which is from Nissan Adv plugin. Dont know what they did different.

i have the full source to uprev so i could check to see how they're doing it, wouldn't have to reverse the PID data then

That would be great. Any chance you can dig through the source and then send me portions of the code that interpret PIDs?

In fact, this screenshot from their program is exactly what I'm looking for. Those values listed on the right side are exactly the ones from the service manual that are supposed to be accessible via the Consult tool. The ones I've decoded so far match the list.

If you can focus on that from the UpRev source that would be perfect.

http://www.the370z.com/attachment.ph...1&d=1406056064

How's this project coming along? I've been pretty busy with other projects, and I have not had a chance to get around to working on anything related to this one.

However, I recently wrote code to produce my own circular progress bar widget, and it can scale down to 16x16 pixels. I'm taking this idea a step further by creating meters similar to the GT-R MFD. The control drawing is ratio-based so the it will be scalable. It is also user configurable as well (colors, dial thickness, etc.).

I envision the ability to create a base program that can read in an XML file, which will determine the layout of the controls. The grid size is nearly unlimited, and it will be limited by the screen size and resolution. We'll see how it goes.

I've only spent about an hour on it, so here is a test bed for the meter user control.
http://www.the370z.com/attachment.ph...1&d=1414697583

This is pretty sweet! I did something similar using python and a PI and displaying the gauges in my head unit, but didn't go to far with it [https://github.com/lcintron/PyO-X]. You guys have much more robust applications, we should create a robust Windows 8 app for display this info, I think we have the talent here (pretty obvious). I am more comfortable in .NET that I am with python so this should be fairly trivial. What do you guys think?