[JAVA] [Adult free study] Acoustic communication on Android

Introduction

I wanted to do indoor positioning with sound waves, but I thought it would be a hassle to buy an ultrasonic speaker, but I was trying to do it, but apparently I found that Android speakers and microphones can cover even the high range. So I tried it.

However, unfortunately, the time was over because the summer vacation period was not enough and ASK (OOK) was done with ultrasound.

What was used

-** MPAndroidChart : A beautiful graph can be drawn without any trouble. Surprisingly, it turned out that even quasi-real-time processing did not work. It's amazing. - AudioRecord : A guy who can process the sound picked up from the microphone at any time. It seems that most models support the sampling rate of 44.1kHz. - AudioTrack : Waveform data can be generated from the speaker as audio. The transmission was full at about 20kHz. Inaudible range (I can hear a little if I have a good ear) - FFT4g **: Kyoto University Oura's FFT package (java ported version), the interface is a little quirky, but it's insanely fast. It may or may not be used by NASA.

Screen capture

From the top, time waveform, envelope, frequency waveform. Even though the name is SoundLocater, it's cute that it can't even position at all.

Where you are saying "yes"

Where I'm saying "Wow"

ASK modulated with 18kHz ultrasound

――It's really beautiful because I put it out with my speaker and pick it up with my microphone ――I could pick up sounds from other devices in a room of about 10m. ――Sounds from other devices are not so beautiful

Try it

――I didn't want to do carrier wave playback or synchronous detection, so I chose ASK. ――But it's still indistinguishable from distance attenuation, so after all it's a play level ――FSK uses some bandwidth, and PSK can be identified on Android, so it's quite difficult. --Recent Android is amazing ――The terminal of a long time ago couldn't stand the processing and died in a few seconds (because I tried to draw in real time). --nova lite I don't die at all. I don't really care about drawing delay

from now on

--I want to detect the beginning exactly ――Is there any good modulation other than ASK? ――I want to put in a simple CSMA / CA-like protocol and avoid each other well.

Implementation

With some excerpts. I don't have enough energy to put the whole thing \ _ (: 3 "∠) \ _

Receiver

The following two are important.

--Use AudioRecord class --Fourier Transform (FFT)

The inverse Fourier transform didn't work and I was quite addicted to it, but in the end it seems that fft.rdft (-1, FFT data) was good.

Also, the following two items were added that were not included in the reference source.

--In order to pass only the desired band, the unnecessary band is set to 0 in the frequency domain. --I converted it to an envelope with the intention of asynchronous detection (it did not reach detection)

        //Creating an Audio Record
        audioRec = new AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLING_RATE, CHANNEL_CONFIG, AUDIO_FORMAT, 2*bufSize);
        audioRec.startRecording();
        isRecording = true;

        //Fourier analysis thread
        fft = new Thread(new Runnable() {
            @Override
            public void run() {
                byte buf[] = new byte[bufSize * 2];
                while (isRecording) {
                    audioRec.read(buf, 0, buf.length);

                    //Endian conversion
                    ByteBuffer bf = ByteBuffer.wrap(buf);
                    bf.order(ByteOrder.LITTLE_ENDIAN);
                    short[] s = new short[bufSize];
                    for (int i = bf.position(); i < bf.capacity() / 2; i++) {
                        s[i] = bf.getShort();
                    }

                    //Creating an FFT class and passing values
                    FFT4g fft = new FFT4g(FFT_SIZE);
                    double[] FFTdata = new double[FFT_SIZE];
                    for (int i = 0; i < bufSize; i++) {
                        FFTdata[i] = (double) s[i];
                    }
                    fft.rdft(1, FFTdata);

                    //Decibel calculation
                    short[] dbfs = new short[FFT_SIZE / 2];
                    for (int i = 0; i < FFT_SIZE; i += 2) {
                        dbfs[i / 2] = (short) (
                                20 * Math.log10(
                                        Math.sqrt( Math.pow(FFTdata[i], 2) + Math.pow(FFTdata[i + 1], 2) )
                                        /dB_baseline
                                )
                        );

                        //★★ Set unnecessary band to 0 in the frequency domain to pass a specific band
                        if ( width/2 < Math.abs(rxFreq-resol*i/2) ) {
                            FFTdata[i] = 0;
                            FFTdata[i+1] = 0;
                        }
                    }

                    //Reverse FFT
                    fft.rdft(-1, FFTdata);

                    //★★ Moving average with absolute value for envelope detection
                    short[] s2 = new short[bufSize];
                    for (int i=16; i<bufSize; i++) {
                        for (int j=0; j<16; j++) {
                            s2[i-16] += (Math.abs(FFTdata[i-j]) * 2.0 / FFT_SIZE) /16;
                        }
                    }

                    updateChart(mChartTime, s);
                    updateChart(mChartTime2, s2);
                    updateChart(mChartFreq, dbfs);

                }
                //Stop recording
                audioRec.stop();
                audioRec.release();
            }
        });
        //Thread start
        fft.start();

reference

It is roughly as follows.

-Let's analyze the frequency on Android --TB-LAB BLOG

This is the FFT.

Transmitter

Compared to the receiving part, the transmitting part was considerably more difficult.

The important things are as follows.

--Use AudioTrack class ――If you modulate with just a square pulse, noise will be generated, so it will be in the time domain.

Ingenuity etc.

--For head detection, 16bit 01 is repeatedly added to the beginning as a preamble. --Carrier wave is 1100 square pulse --ASK (Amplitude-Shift Keying) with 1 pulse of 40ms ――If I used just a square pulse at first, the sound leaked out of the desired band, and I heard noise even at 18kHz (non-audible range). ――If I had to smooth the edges with a raised cosine filter, it became a level that didn't bother me so much.

        //Frequency setting
        int durationPerSymbol = 40;
        int samplesPerT = 4;
        int samplesPerSymbol = (int)( (double)( freq * durationPerSymbol) / 1000.0 );
        samplesPerSymbol = samplesPerT * (int) Math.ceil(samplesPerSymbol / samplesPerT);
        int samplingRate = samplesPerT * freq;

        //Data generation
        Random rnd = new Random();
        byte[] preamble = new byte[]{1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0};
        byte[] data = new byte[64-preamble.length];
        for (int i=0; i<data.length; i++) {
            data[i] = (byte)rnd.nextInt(2);
        }
        int length = preamble.length + data.length;

        //Carrier generation
        byte[] samples = new byte[samplesPerSymbol*length];
        for (int i=0; i<samplesPerSymbol/samplesPerT*length; i++) {
            samples[samplesPerT * i + 0] = (byte) 0x70;
            samples[samplesPerT * i + 1] = (byte) 0x70;
            samples[samplesPerT * i + 2] = (byte) 0x00;
            samples[samplesPerT * i + 3] = (byte) 0x00;
        }

        //AM modulation (preamble)
        int rolloffSamples = samplesPerSymbol/8;
        for (int i=0; i<preamble.length; i++) {
            for (int j=0; j<samplesPerSymbol; j++) {
                double factor = 1.0;

                //Raised cosine filter
                if (j<rolloffSamples) {
                    factor = (1 - Math.cos( Math.PI/(double)rolloffSamples * (double)j )) /2;
                } else if (samplesPerSymbol-rolloffSamples<j) {
                    factor = (1 - Math.cos( Math.PI/(double)rolloffSamples * (double)(samplesPerSymbol-j) )) /2;
                }

                samples[samplesPerSymbol * i + j] = (byte)( (double)(samples[samplesPerSymbol * i + j] * preamble[i]) * factor );
            }
        }

        //AM modulation (data part)
        for (int i=0; i<data.length; i++) {
            for (int j=0; j<samplesPerSymbol; j++) {
                double factor = 1.0;

                //Raised cosine filter
                if (j<rolloffSamples) {
                    factor = (1 - Math.cos( Math.PI/(double)rolloffSamples * (double)j )) /2;
                } else if (samplesPerSymbol-rolloffSamples<j) {
                    factor = (1 - Math.cos( Math.PI/(double)rolloffSamples * (double)(samplesPerSymbol-j) )) /2;
                }

                samples[samplesPerSymbol * (preamble.length+i) + j] = (byte)( (double)(samples[samplesPerSymbol * (preamble.length+i) + j] * data[i]) * factor );
            }
        }

        //Repeat 10 times
        final byte[] txsamples = new byte[10*samples.length];
        for (int i=0; i<10; i++) {
            for (int j=0; j<samples.length; j++) {
                txsamples[samples.length*i+j] = samples[j];
            }
        }

        //AudioTrack constructor
        final AudioTrack mTrack = new AudioTrack(
                AudioManager.STREAM_MUSIC,
                samplingRate,
                AudioFormat.CHANNEL_OUT_MONO,
                AudioFormat.ENCODING_PCM_8BIT,
                txsamples.length,
                AudioTrack.MODE_STATIC
        );

        //Playback complete listener settings
        mTrack.setNotificationMarkerPosition(txsamples.length);
        mTrack.setPlaybackPositionUpdateListener(
            new AudioTrack.OnPlaybackPositionUpdateListener() {
                public void onPeriodicNotification(AudioTrack track) {}
                public void onMarkerReached(AudioTrack track) {
                    if (track.getPlayState() == AudioTrack.PLAYSTATE_PLAYING) {
                        track.stop();
                        track.release();
                        track = null;
                    }
                }
            }
        );

        //Writing and playing waveform data
        track = new Thread(new Runnable() {
            @Override
            public void run() {
                mTrack.reloadStaticData();
                mTrack.write(txsamples, 0, txsamples.length);
                mTrack.play();
            }
        });
        //Thread start
        track.start();

reference

I referred to this area around AudioTrack. Almost as it is.

-2nd Let's emit ultrasonic waves | TONGARISM.COM

Everyone who took care of me

Studying digital communication

-Carrier Digital Communication

Playback relations

-2nd Let's emit ultrasonic waves | TONGARISM.COM -Let's analyze the frequency on Android --TB-LAB BLOG

Recording related

-DSAS Developer's Room: Using Sound 2-Sonic Communication on Android Devices: Preparation-

FFT related

--Java ported version of Dr. Oura's FFT library - Ooura-FFT-Library-by-Other-Language/fft4g.java at master · YSRKEN/Ooura-FFT-Library-by-Other-Language - FFT4g.java --People who are addicted to the above library -Mr. Oura's FFT library addiction --How to disappear completely -Real number FFT / IFFT function --nabe miscellaneous notes

Graph drawing related

-I made a sample of "Sensor data time series graph of real-time update" using MPAndroidChart --Qiita

Recommended Posts

[Adult free study] Acoustic communication on Android
Use serial communication on Android
Try communication using gRPC on Android + Java server
[Android] Notes on xml