C# Scripting - added two new ML+OSD examples

Fixed the problem with ComputerVision API not resolving models/images via HTTP links and created two examples:

Aim Trainer 2D ML Aimbot+ESP

In this one I am showing how to:

• find a window by title
• create ComputerVision that you could use to find images, do ML, OCR and other things
• create SendInput which you could use to simulate inputs
• create OnScreenCanvas which you could use to draw things on the screen
• find object via pre-trained ML-model
• move mouse and click at the center of it
• draw current window position/object position on the screen via OSD (on-screen-display)

Realistically, these three combined are more than enough to create a bot :) All the tools are right there in the box.


Aim Trainer 2D ML Aimbot+ESP ImGui

Verrry similar to the one above, except for a fact that it uses ImGui - immensely powerful drawing library that could be to create UIs as well as draw boxes/text/etc right there on the screen.

Bugfixes/Improvements

• [Scripting] Fixed a problem with CV API not loading HTTP links properly

Send Input - Usb2kbd rework

Usb2Kbd is hardware input emulator. It has been supported by EyeAuras since ~2021. Now you can create your own for approximately 5$ - the only thing that is needed is small device from AliExpress/Amazon. Here is the detailed step-by-step guide (russian / english).

I've also improved technical part and reworked settings window for better user experience.

Bugfixes/Improvements

• [Scripting] Made it so Behavior Tree editor could be shown from Scripts - allows you to create your own custom behavior trees in your mini-apps

ImGui integration EN / RU

Started working on native integration with ImGui - this is alternative approach to script/UI creation in EyeAuras. This should be the simplest one for those who only recently started coding - yes, some boilerplate code is requires to get things started, but after that it is very easy to understand. And ChatGPT can greatly help you.

Finally, it is possible to create truly 1-script mini-apps that incapsulate both UI and automation logic.

Bugfixes/Improvements

• None that time

Bugfixes/Improvements

• [Scripting] Memory reading APIs are now case-insensitive by default
• [Scripting] Memory reading APIs - Added ReadManaged<T> which is slower than Read<T>, but respects MarshalAs and managed structures
• [Scripting] Memory reading APIs are now supporting both x86 and x64 as targets. Keep in mind that your own process is x64 anyways.
• [UI] Fixed a problem with new Folder Editor selection
• [Crash] Fixed nasty crash in a new Folder editor

DMA-integration improvements

EyeAuras uses LeechCore, which is a fantastic library developed by Ulf Frisk, to integrate with DMA-devices out there. De-facto, LeechCore is a standard with which most devices out there are compatible.

For now, that integration is still in alpha-stage, drop me a message if you want to help with testing and are interested in developing something. Realistically, development using FPGA-board does not differ in any way from using ReadProcessMemory - EyeAuras abstracts you from all the complexities of managing LC API, reconnects, error handling, etc. All you have to focus on is writing a code.

Added more info here

Bugfixes/Improvements

• [Scripting] Fixed a problem with FPGA-plate not being properly released upon script disposal

Enabling conditions changes

Made it so only a subset of all Triggers is not available for selection as Enabling Conditions. Yes, that removes some flexibility from the system, e.g. you no longer can use Image analysis triggers. But. This also makes the system a bit simpler to understand, especially for new users.

Enabling conditions always meant to be global On/Off switches. If you want something more dynamic - use Trees/Macros or Auras with Triggers in them.

Folder properties editor rework

I continue to rework existing EA components 1-by-1 to leverage new UI framework capabilities. Newer version of folder properties editor not only loads faster, but is also more flexible.

Legacy version

New version. For the first revision I just wanted to reach feature-parity with existing functionality.

Default properties in Trees/Macros

Made it so you can set Target Window/Input Simulator for each tree/macro separately. Keep in mind that they also respect the normal properties hierarchy. E.g. if you set Target Window on a folder, in which Tree is located, it will respect that. But will also allow you to override that by setting another value on the tree itself.

Bugfixes/Improvements

• Nothing in this version

Bugfixes/Improvements

• [Scripting] Fixed Export to .csproj
• [UI] Fixed crash which in some cases occured during resizing
• [UI] Improved script cancellation logging - show now print the reason why user script got cancelled, e.g. because of Main window activation

Upgrade to .NET 8

.NET is a platform upon which EyeAuras is built. It affects all the tiny bits - startup, memory allocation, data management, literally every small bit of the program is affected by this change.

NET8 is not the latest version (there is already a preview of .NET 10), but platform upgrade is not something you do "just because", there must be a reason. I've picked .NET8 (and not .NET9/.NET10) because only for that specific version there is a prototype of a new memory management mechanism, which, from the current tests, just blows out of the water literally everything else that we had in .NET since its inception.

That is one of the primary characteristics of this new mechanism in comparison to what we have now ("workstation-sustainedlowlatency"). Basically this is the duration of those "hiccups" which tend to happen and are in worst cases become noticeable.

Memory VS CPU

Traditionally, there is always a tradeoff when you're developing a system - you can spend less CPU time, but it will cost you some extra memory OR you can save some memory, but it will increase CPU consumption. EyeAuras has almost always been picking the first option - memory is cheap and cpu cycles are expensive in consumer PCs (purely my opinion). But, aligning to that strategy also means that any improvements in the very core of memory management system should affect us greatly and that memory management mechanism seems to be a perfect fit. We'll see how it goes, but I am expecting to achieve a measurable performance jump in the nearest future.

Bugfixes/Improvements

• [Scripting] Added ScriptContainerExtension prototype - there will be a separate article on this later. For now it is in alpha-stage.

C# Scripting - Advanced - Referencing Scripts from BTs/Macros

That feature allows you to Link from BT/Macro another aura containing C# Script action - this is equivalent of you "referencing" another project in programming. All the classes and types defined in that script will become available to you for use in BTs.

Referencing has existed for many months by now, but was "hidden" from a public eye as it was not fully complete and was not working in a way I wanted it to.

How it that useful?

Now you can have C# class, which will analyze the game - be it via Computer Vision triggers or by directly reading values from memory - and provide BehaviorTree with a data necessary for making a decision. Instead of cluttering the tree with all the checks and having 50 different Variables, you can now have a single class, e.g. TheGame, which will be wrapping up ways of getting the info.

Example of a bot

This is a bot which under the hood uses that same approach which I will be outlining below.

Aura with a Script in it

First, lets take a look at "Shared" aura and files which it has.

TheGame.cs

public sealed class TheGame {
    public TheGame(IFluentLog log){
        Log = log;
    }

    public IFluentLog Log {get;}

    public int IntValue { get; private set; }

    public void Refresh()
    {
        IntValue++; //in real-world apps, this could be the code reading values from Memory or calling Refresh on Triggers
        Log.Info($"Refreshed the state, value: {IntValue}");
    }
}

In that class we have all the scaffolding done and prepared - we initialize triggers OR prepare everything for memory-reading, maybe create some OSD - basically prepare to do the actual work. Whenever Refresh method gets called, the actual data refresh happens. In the example which I am showing, this is happening as a part of a ticking BT. In real-world scenarios this could be on timer, from BT, manually OR even all the methods combined - you're not restricted by anything here, do whatever you want.

Script.csx

[Inject] IAuraTreeScriptingApi AuraTree {get; init;}

Log.Info("Bot is being started!");

var tree = AuraTree.GetBehaviorTreeByPath("./New tree");
var game = GetService<TheGame>(); //create TheGame
tree["TheGame"] = game; //save instance to Variables, making it possible to access it from BTs/Macros

This is how we're propagating TheGame to BT - via Variables. This is the key part that was missing before. From now on, objects passed via Variables should be fully accessible and there are no restrictions on values that are being passed around.

Behavior Tree

Now getting to BT. Here is an example structure which you could use.

Refresh the data

var game = Variables.Get<CheatCrescendo.TheGame>("TheGame").Value;
game.Refresh();

In the first node, we call Refresh, which is expected to refresh game state, actualize it and make it possible to consume in further nodes. Keep in mind, that variable "TheGame" by that point is already filled and points to an instance of TheGame we've created before. Surely you can just have another node in the tree creating that on the first run, but that is up to you to decide. Somehow, you have to create "TheGame" before accessing it here.

Draw OSD

return Run();

public IEnumerator<NodeStatus> Run(){
    using var osd = GetService<IOnScreenCanvasScriptingApi>().Create();

    var game = Variables.Get<CheatCrescendo.TheGame>("TheGame").Value;

    var textOsd = osd.AddHtmlObject();
    while (!cancellationToken.IsCancellationRequested){
        textOsd.Html = $"{game.IntValue}";
        yield return NodeStatus.Success;
    }
}

That is an example of EyeAuras-driven OnScreenDisplay, which can draw whatever you want on the screen. You do not really need it, it is just an example of what you could have. It uses recently-added IEnumerator<NodeStatus> approach, which allows you to "split" node lifecycle in parts. In that case, we're creating OSD on the first run of the node and then simply update it on all consequent runs. As this always happens right after Refresh has been called, your OSD will be tightly in sync with that bot "sees".

That is an example of how you can utilize OSD to better "visualize" decision-making of the bot. Note those colored lines and figures - all of them mean something and having that drawn on the screen immensely happens with debugging. Also it looks cool.

Game logic

And under that selector you can have however many nodes, checking the actual state of the game. Considering all those nodes have access to TheGame object and its methods, you can do whatever you want here. Expose HP as a number that is read directly from memory ? No problem, write that logic in TheGame, call that method from BT Node and you're good to go. Or you can have that same method reading the value from ColorSearch trigger doing color analysis check. Or you can use TextSearch to get that value from UI.

What is also cool is the fact that you can have methods doing something, i.e. you expose a method which uses a bunch of operations to get you from Town A to town B - opens portal, selects and option, clicks on it, etc. Then you simply call that method from BT and it takes you wherever you wanted.

Performance

A whole bunch of performance-related changes. Please report any problems you will notice.

Bugfixes/Improvements

• [Behavior Trees] Added parallel nodes loading prototype - should speed up loading of large trees
• [Scripting] PoeShared.Blazor.Controls that contains default EyeAuras controls is now exposed to scripts by default

Bugfixes/Improvements

• [Behavior Trees] Fixed PopOut crash
• [Behavior Trees] Fixed Action nodes not invoking linked Childs
• [Behavior Trees] Fixed "Toggle Nodes" not working properly
• [UI] Drag/Resize improvement - should feel a bit more "snappy" now
• [DMA] Changed DLL loading order - should fix a problem with LeechCore not initializing properly on some OSes/PCs