Evaluation of Encoder-Only Transformer for Multi-Step Traffic Flow Prediction by Mas Omar, Fitri Yakub, Andika Aji Wijaya, Ahmad Faiz Mohammad, Mohd Nazmin Maslan, Inge Dhamanti, Shi Zhongchao

Subjects: Get full text

Encoding IoT Data: A Comprehensive Review of Image Transformation Techniques by Suat Özdemir, Feyza Yıldırım Okay, Duygu Altunkaya

Subjects: Get full text

Triplet-Style Dynamic Graph Network With Transformer Encoder for Scam Detection in Cryptocurrency Transactions by Min-Woo Nam, Hyeon-Ju Lee, Seok-Jun Buu

Subjects: Get full text

An enhanced network for extracting tunnel lining defects using transformer encoder and aggregate decoder by Bo Guo, Zhihai Huang, Haitao Luo, Perpetual Hope Akwensi, Ruisheng Wang, Bo Huang, Tsz Nam Chan

“…We propose a deep network model utilizing an encoder–decoder framework that integrates Transformer and convolution for comprehensive defect extraction. …”
Get full text

Noise robust aircraft trajectory prediction via autoregressive transformers with hybrid positional encoding by Youyou Li, Yuxiang Fang, Teng Long

“…This study introduces the Noise-Robust Autoregressive Transformer, a novel model that enhances prediction reliability by integrating noise-regularized embeddings within a multi-head attention equipped with hybrid positional encoding. …”
Get full text

The vestibular system implements a linear-nonlinear transformation in order to encode self-motion. by Corentin Massot, Adam D Schneider, Maurice J Chacron, Kathleen E Cullen

“…Although it is well established that the neural code representing the world changes at each stage of a sensory pathway, the transformations that mediate these changes are not well understood. …”
Get full text

Detection and Classification of Abnormal Power Load Data by Combining One-Hot Encoding and GAN–Transformer by Ting Yang, Hongyi Yu, Danhong Lu, Shengkui Bai, Yan Li, Wenyao Fan, Ketian Liu

Subjects: Get full text

Computationally Efficient Single Layer Transformer Convolutional Encoder for Accurate Price Prediction of Agriculture Commodities by Caceja Elyca Anak Bundak, Mohd Amiruddin Abd Rahman, Nurin Syazwina Mohd Haniff, Nur Syaiful Afrizal, Khairul Adib Yusof, Muhammad Khalis Abdul Karim, Md Shuhazlly Mamat, Romi Fadillah Rahmat

Subjects: Get full text

Transformer model with external token memories and attention for PersonaChat by Taize Sun, Katsuhide Fujita

“…This paper introduces a transformer model with external token memory and attention (Tmema) that is inspired by humans’ ability to define and remember each object in a chat. …”
Get full text

Convolutional Swin Encoder by Aditya Majithia, Arthur Paul Pedersen, Michael Grossberg

“…It introduces Convolutional Swin Encoder (CSE), a novel architecture combining Visual Geometry Group Network (VGGNet) and Swin Transformer blocks. …”
Get full text

Contrastive Transformer Network for Track Segment Association with Two-Stage Online Method by Zongqing Cao, Bing Liu, Jianchao Yang, Ke Tan, Zheng Dai, Xingyu Lu, Hong Gu

Subjects: Get full text

Improving the Performance of Heart Disease Diagnosis by Combining the Cochleogram Transformation and Variable Auto-Encoder (VAE) Network by Mahbubeh Bahrayni, Ramin Barati, Abbas Kamali

Subjects: Get full text

Auto-embedding transformer under multi-source information fusion for few-shot fault diagnosis by Bo Wang, Shuai Zhao, Qian Zhao, Yang Bai

Subjects: Get full text

SET: A Shared-Encoder Transformer Scheme for Multi-Sensor, Multi-Class Fault Classification in Industrial IoT by Kamran Sattar Awaisi, Qiang Ye, Srinivas Sampalli

“…Leveraging the transformer architecture, the SET uses a shared encoder with positional encoding and multi-head self-attention mechanisms to capture complex temporal patterns in sensor data. …”
Get full text

Precise length control for large language models by Bradley Butcher, Michael O’Keefe, James Titchener

Subjects: Get full text

Urban Sprawl Monitoring by VHR Images Using Active Contour Loss and Improved U-Net with Mix Transformer Encoders by Miguel Chicchon, Francesca Colosi, Eva Savina Malinverni, Francisco James León Trujillo

“…This study explores the effectiveness of combining Mix Transformer encoders with U-Net architectures to improve feature extraction and spatial context understanding in VHR satellite imagery. …”
Get full text

Medical Report Generation With Knowledge Distillation and Multi-Stage Hierarchical Attention in Vision Transformer Encoder and GPT-2 Decoder by Hilya Tsaniya, Chastine Fatichah, Nanik Suciati, Takashi Obi, Joong-Sun Lee

“…Our approach leverages knowledge distillation with Vision Transformer (ViT) as the image encoder to capture complex visual features, the model benefits from knowledge distillation, transferring knowledge from an ensemble of Convolutional Neural Networks (CNNs) – including VGG16, InceptionV3, and DenseNet121 – to the ViT, ensuring rich and diverse feature extraction. …”
Get full text

Transformer Models for Brazilian Portuguese Question Generation: An Experimental Study by Julia da Rocha Junqueira, Ulisses Brisolara Corrêa, Larissa Freitas

Subjects: Get full text

Structure-Aware and Format-Enhanced Transformer for Accident Report Modeling by Wenhua Zeng, Wenhu Tang, Diping Yuan, Hui Zhang, Pinsheng Duan, Shikun Hu

“…SAFE-Transformer adopts a dual-stream encoding architecture to separately model symbolic section features and heading text, integrates hierarchical depth and format types into positional encodings, and introduces a dynamic gating unit to adaptively fuse headings with paragraph semantics. …”
Get full text

PBC-Transformer: Interpreting Poultry Behavior Classification Using Image Caption Generation Techniques by Jun Li, Bing Yang, Jiaxin Liu, Felix Kwame Amevor, Yating Guo, Yuheng Zhou, Qinwen Deng, Xiaoling Zhao

Subjects: Get full text