iOS視覺-- (10) OpenGL ES+GLSL實現(xiàn)YUV視頻渲染解析

本文借鑒:落影大神--iOS開發(fā)-OpenGL ES實踐教程(一)

本文Demo
前面一篇我們學(xué)習(xí)了OpenGL ES渲染攝像頭錄制的視頻幀的知識,就是將照相機(jī)錄制的CMSampleBuffer轉(zhuǎn)換成CVOpenGLESTexture紋理的再進(jìn)行渲染的過程。如下圖:

CMSampleBuffer轉(zhuǎn)紋理

這篇呢我們將要學(xué)習(xí)使用OpenGL ES渲染YUV視頻幀,原理和渲染照相機(jī)是一樣的。重點在于視頻幀的獲取。這里我使用兩種方式進(jìn)行視頻幀的獲取。這里由于沒有錄制視頻時的回調(diào)方法,我們可以使用一個定時器來代替,這里使用 CADisplayLink

        displayLink = CADisplayLink(target: self, selector: #selector(displayLinkDidUpdate(_:)))
        displayLink.add(to: RunLoop.current, forMode: RunLoop.Mode.default)
        displayLink.preferredFramesPerSecond = 30
        displayLink.isPaused = true
  • 1. 使用 AVPlayerItemVideoOutput 方式獲?。?/h5>
        let videoURL = URL(fileURLWithPath: Bundle.main.path(forResource: "test.mov", ofType: nil)!)
        self.reader = DDAssetReader(videoURL)
        
        let item = AVPlayerItem(url: videoURL)
        player = AVPlayer(playerItem: item)
        let asset: AVAsset = item.asset
        asset.loadValuesAsynchronously(forKeys: ["tracks"]) {
            if asset.statusOfValue(forKey: "tracks", error: nil) == AVKeyValueStatus.loaded {
                let tracks = asset.tracks(withMediaType: AVMediaType.video)
                if tracks.count > 0 {
                    // Choose the first video track.
                    let videoTrack: AVAssetTrack = tracks.first!
                    videoTrack.loadValuesAsynchronously(forKeys: ["preferredTransform"]) {
                        if videoTrack.statusOfValue(forKey: "preferredTransform", error: nil) == AVKeyValueStatus.loaded {
                            let preferredTransform: CGAffineTransform = videoTrack.preferredTransform
                            let preferredRotation = -1 * atan2(preferredTransform.b, preferredTransform.a)
                            NSLog("preferredRotation ----> \(preferredRotation)")

                            DispatchQueue.main.async {
                                item.add(self.videoOutput)
                                self.player.replaceCurrentItem(with: item)
                                self.videoOutput.requestNotificationOfMediaDataChange(withAdvanceInterval: 0.03)
                                self.player.play()
                            }
                        }
                    }
                }
            }
        }

        player.actionAtItemEnd = AVPlayer.ActionAtItemEnd.none
        NotificationCenter.default.addObserver(forName: NSNotification.Name.AVPlayerItemDidPlayToEndTime, object: item, queue: OperationQueue.main) { noti in
            self.player.currentItem?.seek(to: CMTime.zero, completionHandler: { suc in

            })
        }

        mProcessQueue = DispatchQueue(label: "mProcessQueue")

        //kCVPixelFormatType_32BGRA
        videoOutput = AVPlayerItemVideoOutput(pixelBufferAttributes: [String(kCVPixelBufferPixelFormatTypeKey) : kCVPixelFormatType_420YpCbCr8BiPlanarFullRange])
        videoOutput.setDelegate(self, queue: mProcessQueue)

然后通過CADisplayLink的回調(diào)方法來獲取視頻幀:

    @objc func displayLinkDidUpdate(_ sender: CADisplayLink) {
        var outputItemTime: CMTime = .invalid

        // Calculate the nextVsync time which is when the screen will be refreshed next.
        let nextVSync: CFTimeInterval = sender.timestamp + sender.duration
        outputItemTime = videoOutput.itemTime(forHostTime: nextVSync)

        if videoOutput.hasNewPixelBuffer(forItemTime: outputItemTime) {
            var pixelBuffer: CVPixelBuffer?
            pixelBuffer = videoOutput.copyPixelBuffer(forItemTime: outputItemTime, itemTimeForDisplay: nil)
            self.renderView.renderBuffer(pixelBuffer: pixelBuffer)
        }
    }

  • 2. 使用 AVAssetReader + AVAssetReaderTrackOutput 方式獲?。?/h5>
class DDAssetReader: NSObject {

    var readerVideoTrackOutput: AVAssetReaderTrackOutput!
    var assetReader: AVAssetReader!
    var videoUrl: URL!
    var lock: NSLock!
    
    init(_ url: URL) {
        super.init()
        videoUrl = url
        lock = NSLock()
        customInit()
    }
    
    func customInit() {
        let inputOptions = [AVURLAssetPreferPreciseDurationAndTimingKey : true]
        let inputAsset = AVURLAsset(url: videoUrl, options: inputOptions)
        inputAsset.loadValuesAsynchronously(forKeys: ["tracks"]) {
            DispatchQueue.global().async {
                var error: NSError?
                let tracksStatus = inputAsset.statusOfValue(forKey: "tracks", error: &error)
                if (tracksStatus != AVKeyValueStatus.loaded) {
                    NSLog("error = \(error!)")
                    return
                }
                self.processWithAsset(inputAsset)
            }
        }
    }
    
    func processWithAsset(_ asset: AVAsset) {
        lock.lock()
        NSLog("processWithAsset")

        assetReader = try? AVAssetReader(asset: asset)
        
        let outputSettings = [String(kCVPixelBufferPixelFormatTypeKey) : kCVPixelFormatType_420YpCbCr8BiPlanarFullRange]
        
        readerVideoTrackOutput = AVAssetReaderTrackOutput(track: asset.tracks(withMediaType: AVMediaType.video).first!, outputSettings: outputSettings)
        readerVideoTrackOutput.alwaysCopiesSampleData = false
        assetReader.add(readerVideoTrackOutput)

        
        if (assetReader.startReading() == false) {
            NSLog("Error reading from file at URL: %@", asset)
        }
        lock.unlock()
    }
    
    func readBuffer() -> CMSampleBuffer? {
        lock.lock()
        var sampleBuffer: CMSampleBuffer?
        
        if ((readerVideoTrackOutput) != nil) {
            sampleBuffer = readerVideoTrackOutput.copyNextSampleBuffer()
        }
        
        if ((assetReader != nil) && assetReader.status == AVAssetReader.Status.completed) {
            NSLog("customInit")
            readerVideoTrackOutput = nil
            assetReader = nil
            customInit()
        }
        
        lock.unlock()
        return sampleBuffer
    }
}

本文Demo

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請聯(lián)系作者
【社區(qū)內(nèi)容提示】社區(qū)部分內(nèi)容疑似由AI輔助生成,瀏覽時請結(jié)合常識與多方信息審慎甄別。
平臺聲明:文章內(nèi)容(如有圖片或視頻亦包括在內(nèi))由作者上傳并發(fā)布,文章內(nèi)容僅代表作者本人觀點,簡書系信息發(fā)布平臺,僅提供信息存儲服務(wù)。

相關(guān)閱讀更多精彩內(nèi)容

友情鏈接更多精彩內(nèi)容