Send Sequence 2.0

Welcome to the improved Send Sequence action. It now combines all Send action types into one: it can send clicks, type text, move the mouse, and click the mouse, including support for links and anchors.

What’s all the fuss about?

For about 1.5 years, there were 3 separate Send action types: Send Input, Send Text, and Send Sequence.

Each one covered its own specific set of features:

• Send Input was mainly focused on clicking somewhere, such as an image location, anchors, and similar targets.
• Send Text supported several text input methods, such as typing character by character or using the clipboard.
• Send Sequence was always focused on “playing back” longer input sequences, but without the precision of Send Input.

That is now a thing of the past. The redesigned version has the same capabilities as all previous Send action types, and it is much easier to extend with new features and new blocks with any kind of functionality, for example:

• selecting a coordinate
• performing math operations
• using the result to move the mouse

What are the benefits?

The main benefit is that building input sequences is now much easier. For example, if you want to click a specific image, you no longer need to alternate between Send Sequence and Send Input.

I also plan to focus on improving Send Sequence going forward, making it more flexible instead of splitting time across 3 different Send action types.

What should I do with the old Send Input and Send Text actions?

From now on, Send Text and Send Input are effectively in maintenance mode. I will not be adding new options to them or actively working on them anymore.

At some point in the future, they will be fully deprecated and automatically converted into Send Sequence.

You can convert them manually at any time by clicking the Convert to Send Sequence button shown at the top of each Send action. For now, the conversion does not remove the original action, so you can verify that the new one behaves exactly as expected.

Improved image/color/text search: borderless mode

As the title suggests, I added a new option to all of these triggers that allows capturing any window without its border.

In addition, all actions that work by linking/anchoring to the position of a found image will take the missing border into account and adjust coordinates accordingly.

This makes tracking objects in applications that are not running in fullscreen mode easier than ever.

Improved color search

A new option has been added that allows searching for specific pixel blocks of a given color. There is a limited set of supported block sizes, such as 2x2, 4x4, and so on, and the code has been optimized for better overall performance.

Based on my tests, 4x4 blocks give the best results.

Keep in mind that under the hood, this mechanism is very similar to how image search works, so in some scenarios it may still be better to stick with image search. In other cases, though, this new color search mode really shines.

And of course, you can configure anchors/links to the found blocks as well.

Watch the video here

CVATAAT improvements

A fairly detailed documentation page has been added, and it should answer most of your configuration-related questions.

The tool is working and can already be very useful even with its current feature set.

Added support for training with Google Colab

Originally, the tool only allowed training models locally on your own PC using your own hardware. That has obvious limitations: you need to actually have the hardware.

Cloud training makes it possible to use enterprise-grade GPUs such as the A100 for faster and more efficient training in some cases.

It does come with its own limitations, but it can be very useful in many situations. Support is still somewhat limited for now—for example, auto-annotation is not yet available in this mode—but I will continue expanding it.

Bugfixes / Improvements

• [Changelog] Fixed a bug that caused changelogs to be displayed incorrectly, with the text getting “mixed” together
• [Offset, Anchors] Fixed a bug that caused crashes in some cases when adjusting Offset/Anchors in image/text/color search

Bug Fixes / Improvements

• [Overlays] Fixed how offsets work and resolved a crash that could occur in some cases when updating anchors/offsets.
• [Graph / Dependency Viewer] Fixed an issue that caused the graph to display incorrectly due to a missing JS file.

Улучшение возможностей вашего бота с помощью анализа изображений на основе машинного обучения

Машинное обучение, особенно обнаружение объектов, классификация и сегментация, может революционизировать ботов в играх. Представьте себе ботов, которые могут точно определять предметы в игре, врагов и местность, или классифицировать поведение игроков, что приводит к более эффективным и умным решениям в игре. Однако погружение в этот мир может быть сложным. Проблемы? Часто сложно начать, нужно просеять тонну информации, и настройка может быть сложной.

Встречайте инструмент автоматической аннотации CVAT - ваш новый основной инструмент для обучения моделей ML

CVATAAT здесь, чтобы упростить этот путь. Созданный для новичков в ботах, он помогает вам легко подготовить и обучить ваши игровые данные. Это бесплатно, работает на вашем компьютере и сфокусировано на игровых потребностях. Просто подайте ему свои игровые изображения, пометьте их, и CVATAAT делает тяжелую работу, быстро обучая и предоставляя ваши оптимизированные модели, которые вы затем можете использовать с EyeAuras, чтобы получить полную мощь ML-Search trigger.

Это все еще ранняя альфа, поэтому что-то может пойти не так, пожалуйста, сообщайте о проблемах, и вместе мы сможем сделать процесс подготовки моделей как можно проще. Я действительно думаю, что это путь в будущее, с этим инструментом я хотел бы убрать некоторые обломки с дороги.

ИСПРАВЛЕНИЕ ОШИБОК/УЛУЧШЕНИЯ

• [SendSequence] Добавлена поддержка колесика мыши вверх/вниз, доступна только в SendSequence на данный момент
• [Dependencies Viewer/Graph] Теперь использует другой способ отображения аур/зависимостей. Теперь должно быть гораздо более отзывчивым.

Bug Fixes / Improvements

• [Network Message] Fixed a crash caused by a server/client version mismatch

Улучшения автообновления

В Настройках появилась новая опция: "Автоматическая загрузка обновлений". Это гарантирует, что новая версия будет загружена в фоновом режиме, как только она станет доступна, и вы сможете установить обновление гораздо быстрее, чем раньше.

• Вы сохраняете возможность отложить обновление, если сейчас не готовы к установке. Система будет ждать вашей команды!
• Интересно, что в обновлении? Просто нажмите на значок облака, чтобы получить доступ к всплывающему окну с подробной информацией.
• Я усовершенствовал поведение всплывающего окна, чтобы оно было менее навязчивым. Вместо того, чтобы получать уведомления о каждом незначительном обновлении или информации, теперь вы увидите дискретное уведомление рядом с маленьким значком облака в заголовке окна.
• Уведомления об ошибках также были оптимизированы. Короткие сообщения об ошибках появятся рядом с значком облака, а подробные сведения об ошибках будут отображаться во всплывающем окне.
• Журнал изменений также был переписан с нуля. Не стесняйтесь комментировать новый UX, ваше мнение важно!

ИСПРАВЛЕНИЕ ОШИБОК/УЛУЧШЕНИЯ

• [SendInput] [Usb2Kbd] Я ввел новую опцию "CharSet", позволяющую вам выбирать между ANSI и Unicode. Это улучшение направлено на обеспечение совместимости с различными версиями ОС. Настройка по умолчанию - ANSI, что должно быть достаточно для большинства пользователей.

Send Network Message Action / Network Message Trigger Improvements

Infrastructure changes have been rolled out that significantly improve the performance of sending and receiving network messages. This can be used to build a network of interconnected EyeAuras instances that perform actions together, such as remote control, multiboxing, and similar setups.

There are now multiple hubs in each region (RU/EU), and the system will try to choose the one closest to your physical location. This should noticeably reduce latency, especially for RU/CIS users, who previously sometimes used EU servers even for internal traffic. Select the hub closest to you, and the new routing rules will try to use the server in that region that provides the best latency.

As with any infrastructure change, there is always a chance that something might break. If you notice increased latency (or even just higher perceived latency), or something does not work as expected, please let me know.

Latency

In addition to deploying extra servers and adjusting routing, I also changed how network messages are processed, which should further improve overall latency. You can now see the latency for each message directly next to its timestamp. Latency is calculated using the sender's and receiver's local time (in UTC, so the time zone does not matter), which means the value may be somewhat inaccurate if the clocks are not synchronized.

In my own testing, latency dropped from ~100ms to ~20ms after the server-side changes.

US Servers

As far as I know, there are not many users in the US right now, so expanding the infrastructure into a US region is mostly a matter of financial viability. Everything is technically ready, though, so when the time comes, I will add US servers.

Actions / Overlays / Triggers Can Now Be Ignored

Every action, trigger, and overlay can now be Ignored. This is controlled by the checkbox to the left of its name and, effectively, makes the ignored entity behave as if it does not exist.

• For triggers, they are excluded from all evaluations.
• For actions, they are skipped even if the action block is executed.
• For overlays, the overlay is closed and unloaded.

This is especially useful when debugging or configuring something and you want to temporarily disable part of an aura.

P.S. This change does not affect C# scripting.

BUG FIXES / IMPROVEMENTS

• [Text Search] Fixed(?) a crash related to out-of-bounds memory access in Tesseract.

New Windows OCR Text Recognition Method Added

What does this mean?

The TextSearch trigger now includes new options in the "Symbols" dropdown. These let you use Windows-based optical character recognition (OCR) to convert images into text. In testing, this approach proved to be more reliable and performant than the previous Tesseract-based OCR used in EyeAuras.

Please note: to use Windows OCR, you must install the required language pack in Windows. For example, if you want to recognize Spanish characters, the Spanish language pack must be installed on your system.

There is also one limitation that still needs to be addressed: Windows OCR does not support multilingual OCR. In practice, this means it cannot recognize Russian and English characters in the same sentence, for example. There are some well-known workarounds for this, but this method should first be tested as-is, without additional logic layered on top.

A few tips:

• Binary thresholding, which was widely used with Tesseract OCR, is much less effective with Windows OCR. This is one of the reasons the new method is both faster and more reliable.
• Size matters. If OCR returns empty text, try adding scaling through the UI settings. In many cases, even a small image size adjustment can noticeably improve results.
• If you installed a new language and it does not appear in the dropdown, simply restart the application.
• You can read more in the wiki.

Bug Fixes / Improvements

• [Image Effects] Improved the Eyedropper functionality — you can now specify a range that will be applied to the selected color. Exact selection creates an exact match, while narrow/wide applies scaling to create a region.

Bug Fixes / Improvements

• [SendInput] Fixed and improved the driver installation scripts for the virtual driver and input driver. They were no longer working correctly after recent security-related changes.
• [WebUI Overlay] Fixed a rare crash that occurred when using a specific combination of UI controls.

BUG FIXES / IMPROVEMENTS

• [Bug Fix] A window no longer fails to restore after being minimized.
• [Performance] Improved logging performance by reducing the number of WinAPI interactions.
• The Settings and Image Effects windows are now smaller by default, so they fit better on lower resolutions (1280x720).

Bug Fixes and Improvements

• [Speech Overlay] Fixed an issue with region selection when using "Select Region"
• [Image Overlay] Fixed a crash that occurred when the user selected an unsupported file. The overlay now displays a default image to indicate that something went wrong
• [Image Overlay] Added support for the TGA (Targa) format