extracted mfcc features from audio

[ayesha-119]

MFCC Extraction using TensorDSP for Improved Model Accuracy

Overview:

In our ongoing efforts to develop a robust Flutter application for audio-based tasks, I addressed the need for Mel-Frequency Cepstral Coefficients (MFCC) extraction within a Flutter application to enhance model accuracy in audio-based tasks, such as respiratory disease detection. Despite initial success in feature extraction using the Flutter Sound library, the model's performance was unsatisfactory during testing. Upon investigation, it was discovered that the model was trained using features extracted with the MFCC algorithm in Python using the Librosa library. To ensure consistency and improve accuracy, I integrated MFCC extraction directly into the Flutter application using the TensorDSP library. The availability of suitable libraries within the Flutter ecosystem was limited, prompting me to explore native solutions.

Importance of MFCC:

MFCC plays a crucial role in audio-based machine learning tasks by capturing essential features from audio signals. By leveraging MFCC, we can extract discriminative features essential for training and inference in our machine-learning model.
As part of our feature extraction pipeline, we initially relied on the Flutter Sound library in conjunction with a dart:typed_data for audio-to-byte conversion. While this approach provided a foundation for feature extraction crucial for model training and inference, it failed to yield accurate results during testing.

Integration of TensorDSP:

To address this issue, we turned to TensorDSP, a Java-based library known for its robust MFCC extraction capabilities. Leveraging platform channels, specifically Flutter's MethodChannel, we established a seamless communication bridge between Java and Flutter. This allowed us to invoke Java methods from Flutter, enabling us to harness the power of TensorDSP within our Flutter application.

Leveraging Platform Channels:

Platform channels in Flutter facilitate communication between Dart code running in the Flutter app and platform-specific code written in languages such as Java, Kotlin, Swift, or Objective-C. In our case, we utilized a MethodChannel to invoke TensorDSP's MFCC extraction methods written in Java from our Flutter application. This involved defining a channel on both the Flutter and Java sides and establishing a connection to exchange messages.

Setting up the Channel:

1. Flutter Side: We defined a MethodChannel instance in Flutter to communicate with the Java code. This involved specifying a unique channel name and implementing methods to handle incoming and outgoing messages.
2. Java Side: In our Android project, we set up a corresponding MethodChannel instance to receive method calls from Flutter. We registered a MethodCallHandler to process incoming method calls and execute the desired functionality, such as MFCC extraction using TensorDSP.

Addressing Compatibility Issues:

While TensorDSP proved instrumental in achieving our MFCC extraction goals, integrating it into our Flutter project was not without its challenges. Being an older library, compatibility issues arose, requiring thorough troubleshooting and resolution. Despite these hurdles, TensorDSP emerged as the only viable solution capable of meeting our requirements.

Conclusion:

After extensive effort and overcoming various challenges, the integration of TensorDSP for MFCC extraction within the Flutter application is now complete. This enhancement significantly improves the accuracy and reliability of our machine-learning model for respiratory disease detection. I believe this contribution will greatly benefit the project and pave the way for future enhancements in audio-based applications developed with Flutter.

[ayesha-119]

@ruqaiyasattar mam. kindly review this!