I figured out how to get Instance Segmenation color of vehicles

[Emcyz]

in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/Tagger.cpp
there is newly added code at line 70

I thought this code determines color of instance segmantaion
but Actor.GetUniqueID() is not Actor Id in carla simulator
this function is in Unreal Engine which returns Unique ID in Unreal Engine(I guess), not Id in carla
but I could't replace this code to use carla Actor Id
Instead, I changed carla Actor Id to Unique ID

in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorRegistry.cpp at line 80

there is code that determines Id of actor.
IdType Id = ++FActorRegistry::ID_COUNTER;

I removed it and replace with
IdType Id = Actor.GetUniqueID();
then make PythonAPI and I test it to see whether it really works

I run manual_control.py with
print(world.player.id)
print((world.player.id & 0x00ff) >> 0)
print((world.player.id & 0xff00) >> 8)

then I captured screen, checked the RGB value with OpenCV and it was correct!!
ID is 116793, G is 57, B is 200

I hope this can help other guys :)

[DennisKlpng]

Great find!

We don't know yet if there is any way to get the Unreal-Engine ID for a Carla Actor with a specific Carla ID or is there?

The latter would be especially useful for anyone who can't / doesn't want to build Carla from source

[eskildbr]

Hi, a quick question. Did you reverse-engineer or something like that to find the ID when using OpenCV to get the RGB values?

[JosenJiang]

Actually, the correct equation is (blue value << 8) + green value = ID

Sorry, I am still a little confused. According to the equation you gave, the result is not equal to ID. Is there something wrong?

[gogokotsev00]

Actually, the correct equation is (blue value << 8) + green value = ID

Sorry, I am still a little confused. According to the equation you gave, the result is not equal to ID. Is there something wrong?

Yes, an important discovery that I made during my adventure with CARLA (and forgot to mention here :D) was that there is a boundary value = 65536 at which an overload appears. This happens due to the fact that we have 8 bits for each of the R, G and B channels, where the max. value of these 8 bits is 1111 1111₂ = 255₁₀ and for 256₁₀=1 0000 0000₂ we have the overload. If we use the SHIFT LEFT operator, on 256₁₀ we'll then get exactly 65536₁₀. The (B<<8) + G value can be max. 65535 = 255<<8 + 255, which is not the case for the actor's ID. Therefore, in order to know if the calculated result actually does match the actor's ID, I used the following code:

    boundary_value = 65536
    if actor_id > boundary_value:
        actor_id -= math.floor(actorr_id / boundary_value) * boundary_value
   if (B<<8) + G == actor_id:
        return true

At the end of this code snippet, you'll receive the modified actor's ID that should match the (B<<8) + G result, otherwise the pixel does not belong to this specific actor.

[JosenJiang]

Thanks a lot! But according to your method, there may be different actor_ids for the same (B, G) value.
For example:

actor_id_1 = x (x < 65535)
actor_id_2 = x + 65535

I don't know how the UE4 registers an ID for an actor. I am confused whether it will happen.
We can just decode the actor_id to get instance segmentation color. But we can not decode instance segmentation color to get the actor_id.

I try to use A channel like this:

But all values in A channel I get is 255. I have not found solutions.

[gogokotsev00]

Thanks a lot! But according to your method, there may be different actor_ids for the same (B, G) value. For example:

actor_id_1 = x (x < 65535)
actor_id_2 = x + 65535

I don't know how the UE4 registers an ID for an actor. I am confused whether it will happen. We can just decode the actor_id to get instance segmentation color. But we can not decode instance segmentation color to get the actor_id.

I try to use A channel like this: But all values in A channel I get is 255. I have not found solutions.

Yes, when there are identical actor IDs you have to see the red channel and decide whether this is the type of actor you need. Channel A can be omitted for this task, it does not play a role in it.

I couldn't understand what you mean by decoding the instance segmentation colour, probably that we cannot get the correct actor ID from the colours, well that's correct, I also don't know if there is a way.

[JosenJiang]

Yes, that's what I mean. In order to get the correct actor ID from the colors, I try to use G, B and A channels to encode an ID greater than 65535. But I always get 255 from A channel.
Thanks any way.

[Mofeed-Chaar]

many thanks for your information.
my question is, how could I get a Unique id using Python code?????

[Gowtham-Raj-M]

Hi @Mofeed-Chaar, Did you find solution to get a Unique id using Python code?

[Mofeed-Chaar]

not yet
i will try to find a way to run a code of c++ inside a python code

[Gowtham-Raj-M]

Hi @Mofeed-Chaar , Actually i tried different approaches to get color code for objects in Carla, but not able to get the value. Then i tried with above mentioned approach with carla built from source , now i am getting color code for each object.

[DennisKlpng]

Hi, I didn't find any solution to get it directly, what you can do if you can't build from source and if you can do some sort of post-processing (so, depends on your use case):

• move all actors away, afterwards move each actor (by Carla ID) separately directly in front of one instance segementation camera (setting velocity to 0, deactivating physics is beneficial)
• take one frame, save the image + the actor of the player you moved, move the actor away again
• Repeat until you have one instance segmented image for each actor
  When counting the number of pixels with the correct actor class, you should have an association between color and Carla ID for this simulation run

Doesn't work for every use case, but when you just record data and can do post-processing it was reliable enough for me

[venitws2]

in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/Tagger.cpp there is newly added code at line 70

I thought this code determines color of instance segmantaion but Actor.GetUniqueID() is not Actor Id in carla simulator this function is in Unreal Engine which returns Unique ID in Unreal Engine(I guess), not Id in carla but I could't replace this code to use carla Actor Id Instead, I changed carla Actor Id to Unique ID

in carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Actor/ActorRegistry.cpp at line 80 there is code that determines Id of actor. IdType Id = ++FActorRegistry::ID_COUNTER;

I removed it and replace with IdType Id = Actor.GetUniqueID(); then make PythonAPI and I test it to see whether it really works

I run manual_control.py with print(world.player.id) print((world.player.id & 0x00ff) >> 0) print((world.player.id & 0xff00) >> 8)

then I captured screen, checked the RGB value with OpenCV and it was correct!! ID is 116793, G is 57, B is 200

I hope this can help other guys :)

I think this is a very easy to implement solution for the people who are currently working on the CARLA simulator and using the instance segmentation camera because it just replaces a single line of code. However, the reason why the CARLA team still did not solve this issue by using this method might be due to changing the ID assignment mechanism completely. Anyway, I used this method and it solved my problem. Keep in mind that you need to rebuild both the server and client sides from the source after doing this.