gnueabihf version of AW closed source

[jonsmirl]

These AI kernels would run 50% faster if I were able to compile them with gnueabihf calling conventions.  It is a major pain to mix gnueabi and gnueabihf on the same system. Life would be far simpler if AW would provide a gnueabihf version of their libraries.

A huge downside to mixing gnueabi and gnueabihf is that you end up with two copies of all of the system libraries in memory at the same time. For example you have to have two versions of the C run-time, etc, etc.

I've done more benchmarking and it appears that not having gnueabihf hurts AI core performance a lot. On large models like inception it is 50%. On a small model like squeezenet it is 300%.  Not having hf is preventing register passing via the FPU registers.

Rasp3b has same quad core Cortex-A7, armv7l as the V5 does. I should be able to get the same benchmark results or very close on the V5. Only significant difference is gnueabihf on Rasp3b.

squeezenet_v1.1  
    v5 298ms rasp3b 94ms
inception_v3
    v5 3592ms rap3b 2074ms

[given]

Dear jonsmirl, your testing result is very valuable. And we know the gnueabihf can promote the performance of the floating point arithmetic，which is the key kind of arithmetic of AI or image processing.But just like what you said, life would be far simpler if AW would provide a gnueabihf verson of their libraries. We are working hard to build up a gnueabihf version of V5 board system now...And if we done, we will let you know at the first time.

[jonsmirl]

I believe I can work around this by cross compiling the AI core with gnueabihf and then statically linking it to the C runtime libraries. These cores only call out to a few libraries so this is not a huge problem. There is no problem mixing gnueabi and gnuebihf programs on a system. The problem is in mixing the shared libraries.

There is a mistake in the previous performance numbers. I compared to a rasp pi 3 which I thought was A7, but it is not, it is A53. The rasp pi 2 is A7. So it was not a fair comparison.

I sent email to the lindini support email, I believe I have located someone who will write an AI kernel driver for the CVE hardware. But Allwinner has to release the low level documentation for it or the driver can't be written. Currently there is not enough info available to even determine if writing a general purpose driver is possible.

gaofeng<gaofeng@allwinnertech.com> is listed as the author of the CVE kernel module.

[given]

Yes, we have receive your e-mail, and I have told my boss about your suggest. But CVE/EVE is the core of V5, and it is not an easy work to persuade AW to release its low level documentation. And we have told our boss about your suggestion, and he will try to contact AW and talk about it.

[jonsmirl]

If Allwinner would release the tools promised in the manual, we wouldn't need to write our own driver.

9. Classifier visual design, providing a complete set of training, debugging, testing and evaluation tools

1.2.1 EVE Functional Description
1.360p detection speed>30FPS (working frequency >300MHZ)
2. Support classic HAAR feature classifier detection, the number of features up to 3200
3. Support maximum resolution 4K input and internal zoom to support region of interest detection
4. Support 4-channel integral map calculation, processing 1.3 billion features per second
5. Support 3-channel feature calculation
6. Support user-defined target size, support 432 different kinds of detection frames
7. Supports minimum 64x64 pixel single image detection
8. Low power consumption, 360p full picture detection <57mw
9. Classifier visual design, providing a complete set of training, debugging, testing and evaluation tools
10. Customizable classifiers to support arbitrary small deformation rigid target detection (face, vehicle, license plate, pedestrian,

[given]

As I know, that classifier is based on OpenCV, you can train your OpenCV classifier on PC, and then covert it to the format that the EVE hardware can use by a convert tool (has not released).

[jonsmirl]

We can probably work with whatever Allwinner releases. With the current software we have today it is unclear if we can build a sellable system. Allwinner has made a good set of tools for automating parking lots, but not very useful to a company that does not sell parking lot cameras.

There is always a trade off. Keep the technology closed. Then Allwinner has to supply all of the tools, all of the support, fix all of the bugs. And if they don't do that then their customers can't ship and you don't sell many chips. Keeping tech closed comes with a very high support cost.

Or open up the technology. Then sometimes you get lucky and a person will write a popular tool that uses your hardware. And that results in sales from places you never previously thought your chip would be useful for. This doesn't always happen, but when it does the payoff can be huge. Consider the effect of Arduino. Allwinner has an unusual chip with the V5. If they were to release more info on EVE/CVE AI researchers would be able to make use of it. Many of those AI researchers are very smart people and they will write excellent software for the V5.

Here's an example. The EVE/CVE hardware can almost certainly do very accurate spoken hot word detection for services like Alexa. But with the tools I have available, I can't program it to do this. This is an application of the hardware that Allwinner has not considered and it's an application that could potentially sell a lot of chips.