Error with login

[nikshriv]

Thanks for your work on this. Tried running the server but received this error when I tried to log in to the web app:

Error: new controller: login: user version error

Does this have to do with 2 factor authentication?

[unixpickle]

Yeah, the recent 2FA switch breaks logins. As a temporary fix, use the login_2fa program in the root of this repo to get a blob of JSON representing you session. Instead of passing login credentials to the server, pass -sessinfo "blobofjson" to explicitly have the server use the session you just created.

This is a temporary workaround, and you'll have to login to a new session once a week with login_2fa as far as I know. Longer term the server will likely have to have some login screen where you enter a verification code once a week.

[nikshriv]

Thanks so much. I'll give it a try.

[nikshriv]

I gave it a try yesterday. I was able to obtain the json from the server and started the server main.go. Then I tried to open the web app in chrome and got stuck in a login loop. Any ideas?

[unixpickle]

What does "login loop" mean exactly? By the way, don't pass the email or the password arguments anymore once you use sessinfo.

[nikshriv]

By login loop I mean that when I try to open the web app on port 8080 using a web browser, it asks me to login. When I try to log in using my cync account credentials, it fails to log in and returns me to the login box. I have only been passing the sessinfo when I start the server.

[unixpickle]

Ah understood! Passing the -no-auth flag should fix that issue. For background, the web app itself has a separate login so that if you expose it on the internet, random people can't use it and mess with your lights. For most use cases it can be disabled, and that should probably be the default.

If you want to use HTTP basic auth, you can pass -web-password XYZ where XYZ is the password you want to use.

[nikshriv]

Thanks... I tried the -no-auth flag and it no longer requires a login, so that issue is fixed. However, the server shuts down immediately with this error:

invalid character 'a' looking for beginning of value
exit status 1

The webpage shows this error:

TypeError: Failed to fetch

Could this have to do with the fact that I only have c by ge switches and no bulbs?

[jandaj]

Running into the same issue. I haven't looked into it much, but it has to do with the gelighting server complaining about bad authorization [token] from session info when opening the tcp connection to get the status of the devices.

[kylekyle]

Same Failed to fetch error here.

[unixpickle]

So to respond to @nikshriv -- I only have GE bulbs so I've never tested this with switches.

@jandaj do you see the same error @nikshriv saw with:
invalid character 'a' looking for beginning of value

That sounds like a JSON parsing error rather than an error with the TCP server. In fact, it is possibly an error parsing the sessinfo itself if somehow the data was passed wrong or not escaped correctly on the commandline, i.e. could be this line https://github.com/unixpickle/cbyge/blob/main/server/main.go#L327

I can make a branch that will provide more info on fatal errors so that @nikshriv can debug further.

(Also sorry that I closed and reopened the issue. I fat fingered the close button)

[unixpickle]

Btw, possibly the JSON parsing error is because of how sessinfo is passed--the only time the server exits with a fatal error is on the line I posted where it tries to parse sessinfo.

I had suggested passing -sessinfo "blobofjson", but the JSON presumably contains quotes, so you should probably instead do -sessinfo 'blobofjson' (i.e. use single quotes around the JSON). Or put the JSON in a file info.json and pass -sessinfo $(cat info.json)

[nikshriv]

This is definitely a JSON unmarshal problem that occurs in the GetDeviceProperties function. The problem is with my account. There is a device that is empty and therefore has no bulbArray. I deleted it this morning and will test it later today. A fix for this issue would be to delete devices from the returned JSON if there is no bulbArray. I am not familiar with Golang, but trying to teach myself. Not sure how to accomplish this.

[unixpickle]

Ah understood. I can try making that fix once you confirm that deleting the device worked. JSON parsing in Go is a bit finicky for sure.

If the error came from GetDeviceProperties, I'd have expected the error to include get device properties in the message, and I would expect the server not to cash but rather to return the error to the client. If that's not the case, there must be a bug in my error handling logic somewhere.

[nikshriv]

Well, it turns out that modifying GetDeviceProperties to exclude the device entries with no bulbArray did not fix the problem. I modified Login_2fa to attempt device enumeration after logging in by calling Devices() with my sessinfo. The following error occurs:

get device properties: json: cannot unmarshal number 1924033518009 into Go struct field .bulbsArray.switchID of type uint32

It looks like my switchId number is too large to fit into a uint32. Looking through your code, it looks like you use the uint32 value to create a packet when sending commands and getting status of the switch, but since my switchID is too big to be represented as a uint32, this will not work for my devices. I am curious to know how you figured out which bytes and encoding you used and how you were able to figure this out. I have been staring at the packets from my phone app that I gathered using a pcap utility, and I cannot figure out how they are specifiying the switch ID. I

[unixpickle]

Oh wow, thanks for digging into this!

I am definitely surprised that a switch ID can be more than 32 bits, since it appeared that a device's switch ID was the first four bytes of every packet after the big endian length field.

In particular, every packet in this protocol appears to be a byte type field followed by a four byte big endian length field, and most packets are then followed by a four byte big endian device ID (which is the switch ID of some device). I'm curious if you are able to observe this pattern in the packets you gathered. Or you could post the first few bytes of one of the packets so I can take a look--make sure it isn't an auth packet though to avoid revealing sensitive info.

One hack to try to work around this is to change switch ID to uint64, and then cast it to uint32 (syntax is uint32(x)) when using it elsewhere. Maybe those higher bits aren't important or encode some other info? Or maybe something much more complicated is going on than I gathered in my initial exploration of this protocol.

I may also take a look at the smali code from the android app. It has some clues as to how these packets are encoded, so maybe I can see what it does with the switchID.

[unixpickle]

I have been thinking about this more, and it makes a lot of sense now. All of the C by GE devices I own have WiFi support built-in, meaning they have their own "switches". Sometimes, a device won't have WiFi support, so it is only connected to WiFi through some other device. The TCP protocol actually highlights this: the packets that start with the 32-bit switch ID are specifically called "pipe" packets, and these packets do carry a separate 16-bit device index that is the actual destination device. Presumably a device receives a pipe packet through its switch, and forwards the message to the specified device index via a local mesh network.

The above observation is really neat, and suggests a way we could make this codebase more robust to non-WiFi devices--or even devices which temporarily disconnected from WiFi. In particular, I can relax my protocol implementation to talk to devices through any SwitchID, and maybe round-robin through SwitchIDs for each device until it can be reached in case some devices can't talk to each other through a mesh network.

The SwitchID you posted is 1924033518009, and actually looks like all of my devices' DeviceIDs rather than their 32-bit SwitchIDs (a deviceID typically ends in a 3 digit device index, and 009 looks like an index). My guess is that 1924033518009 is a device ID, which is either filled in as a placeholder when this device has no WiFi support, or even more interestingly could indicate a neighboring DeviceID that this device is connected to and can be reached through. I am really curious to see the entire contents of your device properties JSON to understand more about what these fields mean for devices with no built-in WiFi support.

I pushed a commit (facf692) to ignore the error you were seeing from GetDeviceProperties() from now on. I'm not exactly sure why the server is returning text that isn't valid JSON, but now it won't crash device enumeration.

I also created a branch called log-json which you can use to see the exact JSON the server is sending back inside the bulbs array. This should help us figure out exactly how SwitchID is behaving (e.g. if it's a reference to another device in the same bulbs array). Try running git pull && git checkout log-json to go to the log-json branch, and then try running your modified 2FA login script again with device enumeration. This should make it much easier to debug what is going on, and you could potentially post JSON for your device properties so we could understand how the topology is reflected in the output.

[nikshriv]

Thanks again for your work on this...I make a node.js server to connect to the TCP server and have been listening to messages coming from the server. This is an example packet that I believe is an update on my switch state. I can confirm that my deviceID starts from the 6th byte (index 5) in this packet and is a UInt32 number.

83 00 00 00 25 12 fb 5d 14 48 78 00 7e 00 00 00 00 fa db 13 00 00 00 00 19 00 19 00 db 11 02 01 00 00 00 00 00 00 00 00 0f 7e 43 00 00 00 1a 12 fb 5d 14 01 01 06 05 00 10 19 00 00 00 00 00 00 01 00 19 00 00 00 00 00 00

I noticed the same thing about the switchID that you pointed out. The last 3 digits of my switchID appears to be a 3 digit index. My deviceID and switchId are the same, but it sounds like your switchID does not include the index. Both IDs start with the same Uint32 value as the "ID" in the JSON obtained from GetDevices(). Here is an example device properties JSON:

{
"bulbsArray":[
{"deviceType": 48, "switchID": 1384828557022, "displayName": "Master Deck Door", "deviceID": 1384828557022…}
],
"sceneArray":[],
"switchIsAutoUpdate": false,
"schedules":[],
"deviceIdRecord": 1,
"admin":{
"emailAddress": "xxxxxxxx@gmail.com",
"lastUseDate": "2020-07-16 23:45:10",
"userID": 0,
"createDate": "2020-07-16 23:45:10",
"username": ""
},
"groupsArray":[],
"lastUseDate": "2020-07-16 23:45:10",
"placeName": "Switch Place",
"version": "1.0",
"createDate": "2020-07-16 23:45:10"
}

[nikshriv]

This the set of packets I receive when I turn on my Front Door switch from the Alexa app.

83 00 00 00 25 72 ae 64 5a 2d d5 00 7e 00 00 00 00 fa db 13 00 00 00 00 18 00 18 00 db 11 02 01 01 64 00 00 00 00 00 00 72 7e

43 00 00 00 1a 72 ae 64 5a 01 01 06 05 00 10 18 01 64 00 00 00 00 01 00 18 00 00 00 00 00 00 83 00 00 00 25 12 fb 58 bd 58 8a 00 7e 00 00 00 00 fa db 13 00 00 00 00 18 00 18 00 db 11 02 01 01 64 00 00 00 00 00 00 72 7e 43 00 00 00 1a 12 fb 58 bd 01 01 06 1d 00 10 18 01 64 00 00 00 00 01 28 1c 00 00 00 00 00 00

83 00 00 00 25 12 fb 5d 14 4b 7c 00 7e 00 00 00 00 fa db 13 00 00 00 00 18 00 18 00 db 11 02 01 01 64 00 00 00 00 00 00 72 7e

43 00 00 00 1a 12 fb 5d 14 01 01 06 06 00 10 18 01 64 00 00 00 00 01 18 19 00 00 00 00 00 00

[unixpickle]

When you connect to the TCP server via your Node.JS app, you are not intercepting the actual packets your phone would need to send to the server to turn on/off the devices. You are just observing packet types 4 and 8, which are "sync" events (I think they might carry some device status, but they aren't actions that can change the lights). Here are the packet types from the disassembled Android app:

.field public static final TYPE_OUTER_DISCONNECT:I = 0xe
.field public static final TYPE_OUTER_EVENT:I = 0xc
.field public static final TYPE_OUTER_LOGIN:I = 0x1
.field public static final TYPE_OUTER_PING:I = 0xd
.field public static final TYPE_OUTER_PIPE:I = 0x7
.field public static final TYPE_OUTER_PIPE_SYNC:I = 0x8
.field public static final TYPE_OUTER_PROBE:I = 0xa
.field public static final TYPE_OUTER_SET:I = 0x3
.field public static final TYPE_OUTER_SETPWD:I = 0x5
.field public static final TYPE_OUTER_SUBSCRIPTION:I = 0x9
.field public static final TYPE_OUTER_SYNC:I = 0x4

Note that packet type 7, PIPE, is what seems to actually tell devices what actions to perform.

[unixpickle]

On a related note, I merged a big change into the main branch that decouples devices from their SwitchIDs. I can't fully test this on my own network (since all of my devices have their own switches) but I can confirm that the app is able to control and get statuses of devices through other devices' switch IDs, indicating that communicating over the mesh network is working. In my setup, this will make everything more reliable, since I often find that one or two lights temporarily disconnects from WiFi.

This fix may be enough for you to now successfully use the app, although I'm guessing the exact types of packets you need to control your devices will be different than the types of packets I use to control my bulbs. Without buying one of these devices, it would be pretty hard for me to reverse engineer the protocol on my own (although the protocol may be quite similar to the one used for my bulbs).

If you want to take a stab at reverse engineering the protocol specific to your devices, you will probably want to use Wireshark or similar to intercept communications between your phone and the TCP server (and you'll need to turn off Bluetooth on your phone to force it to use the TCP protocol). What I did was a bit more complicated but also more reliable:

1. Run a proxy on my own server for the TCP protocol
2. Disassemble the Android app & changed the hostname of the TCP server to my own server
3. Recompile and install the modified app on my phone.

This is likely quite difficult to do without an Android phone, and honestly was pretty difficult to get right even with an Android phone.

[nikshriv]

I was able to successfully login and enumerate my devices with the new change. Unfortunately, still unable to use the web app and still get the same error:

invalid character 'a' looking for beginning of value
exit status 1

I have been using a packet capture app on my phone and this is the packet I see when turning on a switch from the Cync app:

73 00 00 00 1f 72 ae 7a 4d 82 99 00 7e 65 00 00 00 f8 d0 0d 00 00 00 00 00 00 08 00 d0 11 02 01 00 00 c8 7e

This looks just like a packet from your NewPacketSetDeviceState function. The 08 appears to refer to the "index" at the end of my deviceID, and the "72 ae 7a 4d" refers to a different switch in my house. I'm assuming the app connects to an available switch and then uses the bluetooth mesh to deliver the command to the device with index 08. This appears to be just like your bulbs. However, when I try to replay this packet from my node app, the switch does not change state. I have also tried changing the switchID to refer directly to the switch I'm trying to communicate with but that doesn't work either. I have fiddled with the Uint16 number that follows the deviceID, and that doesn't work. I see that you are passing the value 123 to fill these two bytes. What exactly is the "seq" for? Any ideas on what I may be doing wrong?

[unixpickle]

@nikshriv could you try pulling the code one more time and running the website server again? It seems we are quite close to getting this to work.

The fact that your packet looks similar to my packets is quite promising! The seq number can be pretty much arbitrary, and 123 was basically a random choice that would be recognizable while looking at packet dumps.

Note that you can't replay just the SetStatus packet--you must also replay the auth packet first. You may find it easier to create a PacketConn directly, use its Auth method, and then construct and send a packet via NewPacketSetDeviceStatus(). If that works, it would be quite a good sign!