[Version 3] Add linux perf functionality to cover AMD and similar

[mp15]

This pull request creates an interface to linux perf through python C bindings. This allows us to measure power usage on platforms which do not support Intel RAPL but do have some machine specific registers that linux perf allows us to use to measure consumption.

This pull request is initially submitted as a draft to allow folks to have a look at the structure of the integration and make suggestions. At the moment the whole hardware measurement interface is a bit intel power gadget based and we may with to consider revising this.

[mp15]

On my todo list:

• Figure out how to get python C extensions and pytest playing nicely. When extensions are installed into the build directory they work but when you try and run from the source directory they do not.
• Add actual measurement for RAM power consumption, both RAPL and perf can do this.

[benoit-cty]

Hello, thank you very much !

Did you write all the C code ? It use low level Linux kernel API to get power usage ?

[benoit-cty]

Hello @mp15 , your contribution, is very interesting, do you know if you could finish it soon ?
How can we help you ?

[mp15]

Okay I've made a little progress with this. It should now not completely mess things up on non-linux systems.

[benoit-cty]

Hello, thanks for this contribution.
When trying your fork, I have this error:

[codecarbon WARNING @ 08:31:17] <class 'Exception'>
Traceback (most recent call last):
  File "/media/data-nvme/dev/src/CODE_CARBON/codecarbonmp15/codecarbon/core/util.py", line 18, in suppress
    yield
  File "/home/ben/.pyenv/versions/3.9.9/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/media/data-nvme/dev/src/CODE_CARBON/codecarbonmp15/codecarbon/emissions_tracker.py", line 385, in start
    hardware.start()
  File "/media/data-nvme/dev/src/CODE_CARBON/codecarbonmp15/codecarbon/external/hardware.py", line 190, in start
    self._hw_measurement_interface.start()
  File "/media/data-nvme/dev/src/CODE_CARBON/codecarbonmp15/codecarbon/core/cpu.py", line 200, in start
    self._perfinterface.start()
PermissionError: [Errno 13] Permission denied

I'm on Ubuntu 20.04 with an AMD CPU.

I saw that normal user can read this file:

-r--r--r-- 1 root root 4096 nov.  13 07:11 /sys/devices/system/cpu/cpu0/topology/physical_package_id
ben@rig: ~ >cat /sys/bus/event_source/devices/power/events/energy-pkg.scale 
2.3283064365386962890625e-10
ben@rig: ~ >cat /sys/bus/event_source/devices/power/events/energy-pkg.unit 
Joules

What can be the problem ?

[benoit-cty]

Could you explain how to test this ? It does not start on my computer (Ubuntu 22.04 on AMD GPU)

[mp15]

Hello, thanks for this contribution. When trying your fork, I have this error:
I saw that normal user can read this file:
/sys/devices/system/cpu/cpu0/topology/physical_package_id
/sys/bus/event_source/devices/power/events/energy-pkg.scale
/sys/bus/event_source/devices/power/events/energy-pkg.unit
What can be the problem ?

Unfortunately the kernel perf interface is restricted to processes with CAP_PERFMON or root access.

[SaboniAmine]

Hello @mp15, should we expect some follow up work on this PR ? Would you need some help ?
Otherwise, and if you lack time, how could we finish your work ?

[benoit-cty]

Hi mp15, I saw you add AWS Graviton, does it works with your PR ?

[benoit-cty]

I'm sorry, I think we will have to close this PR without more information on it.

@mp15 Can you help us better understand it ?