boofcv.android.camera2

Class VisualizeCamera2Activity

java.lang.Object

android.content.Context

android.content.ContextWrapper

android.view.ContextThemeWrapper

android.app.Activity

boofcv.android.camera2.SimpleCamera2Activity

boofcv.android.camera2.VisualizeCamera2Activity

All Implemented Interfaces:

android.content.ComponentCallbacks, android.content.ComponentCallbacks2, android.view.KeyEvent.Callback, android.view.LayoutInflater.Factory, android.view.LayoutInflater.Factory2, android.view.View.OnCreateContextMenuListener, android.view.Window.Callback

public abstract class VisualizeCamera2Activity
extends SimpleCamera2Activity

Extension of SimpleCamera2Activity which adds visualization and hooks for image processing. Video frames are automatically converted into a format which can be processed by computer vision. Optionally multiple threads can be generated so that video frames are processed concurrently. The input image is automatically converted into a Bitmap image if requested. If multiple threads are being used the user can toggle if they want visualization to be shown if an old image finished being processed after a newer one. Must call startCamera(ViewGroup, TextureView) in onCreate(). To customize it's behavior override the following functions:

• setThreadPoolSize(int)
• setImageType(boofcv.struct.image.ImageType, boofcv.alg.color.ColorFormat)

Useful variables

• imageToView: Matrix that converts a pixel in video frame to view frame
• displayView: Where visualizations are rendered in.
• targetResolution: Specifies how many pixels you want in the video frame
• stretchToFill: true to stretch the video frame to fill the entire view
• showBitmap: If it should handle convert the video frame into a bitmap and rendering it
• visualizeOnlyMostRecent: If more than one thread is enabled should it show old results

Configuration variables

• targetResolution
• showBitmap
• stretchToFill
• visualizeOnlyMostRecent

See Also:

SimpleCamera2Activity

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	VisualizeCamera2Activity.BoofImage Data related to converting between Image and BoofCV data types All class data is owned by the lock
class	VisualizeCamera2Activity.DisplayView Custom view for visualizing results

Nested classes/interfaces inherited from class boofcv.android.camera2.SimpleCamera2Activity

SimpleCamera2Activity.CameraState

Field Summary

Fields

Modifier and Type	Field and Description
protected android.graphics.Bitmap	bitmap
protected java.lang.Object	bitmapLock
protected byte[]	bitmapTmp
protected VisualizeCamera2Activity.DisplayView	displayView
protected android.graphics.Matrix	imageToView Stores the transform from the video image to the view it is displayed on.
protected java.lang.Object	lockTiming
protected boofcv.misc.MovingAverage	periodConvert
protected boolean	showBitmap
protected boolean	stretchToFill
protected int	targetResolution
protected android.view.TextureView	textureView
protected java.util.concurrent.ThreadPoolExecutor	threadPool
protected java.util.concurrent.LinkedBlockingQueue	threadQueue
protected long	timeOfLastUpdated
protected static int	TIMING_WARM_UP
protected int	totalConverted
protected android.graphics.Matrix	viewToImage
protected boolean	visualizeOnlyMostRecent

Fields inherited from class boofcv.android.camera2.SimpleCamera2Activity

cameraToDisplayDensity, displayMetrics, mTextureView, mView, verbose, viewHeight, viewWidth

Fields inherited from class android.app.Activity

DEFAULT_KEYS_DIALER, DEFAULT_KEYS_DISABLE, DEFAULT_KEYS_SEARCH_GLOBAL, DEFAULT_KEYS_SEARCH_LOCAL, DEFAULT_KEYS_SHORTCUT, FOCUSED_STATE_SET, RESULT_CANCELED, RESULT_FIRST_USER, RESULT_OK

Fields inherited from class android.content.Context

ACCESSIBILITY_SERVICE, ACCOUNT_SERVICE, ACTIVITY_SERVICE, ALARM_SERVICE, APP_OPS_SERVICE, APPWIDGET_SERVICE, AUDIO_SERVICE, BATTERY_SERVICE, BIND_ABOVE_CLIENT, BIND_ADJUST_WITH_ACTIVITY, BIND_ALLOW_OOM_MANAGEMENT, BIND_AUTO_CREATE, BIND_DEBUG_UNBIND, BIND_EXTERNAL_SERVICE, BIND_IMPORTANT, BIND_NOT_FOREGROUND, BIND_WAIVE_PRIORITY, BLUETOOTH_SERVICE, CAMERA_SERVICE, CAPTIONING_SERVICE, CARRIER_CONFIG_SERVICE, CLIPBOARD_SERVICE, COMPANION_DEVICE_SERVICE, CONNECTIVITY_SERVICE, CONSUMER_IR_SERVICE, CONTEXT_IGNORE_SECURITY, CONTEXT_INCLUDE_CODE, CONTEXT_RESTRICTED, DEVICE_POLICY_SERVICE, DISPLAY_SERVICE, DOWNLOAD_SERVICE, DROPBOX_SERVICE, FINGERPRINT_SERVICE, HARDWARE_PROPERTIES_SERVICE, INPUT_METHOD_SERVICE, INPUT_SERVICE, JOB_SCHEDULER_SERVICE, KEYGUARD_SERVICE, LAUNCHER_APPS_SERVICE, LAYOUT_INFLATER_SERVICE, LOCATION_SERVICE, MEDIA_PROJECTION_SERVICE, MEDIA_ROUTER_SERVICE, MEDIA_SESSION_SERVICE, MIDI_SERVICE, MODE_APPEND, MODE_ENABLE_WRITE_AHEAD_LOGGING, MODE_MULTI_PROCESS, MODE_NO_LOCALIZED_COLLATORS, MODE_PRIVATE, MODE_WORLD_READABLE, MODE_WORLD_WRITEABLE, NETWORK_STATS_SERVICE, NFC_SERVICE, NOTIFICATION_SERVICE, NSD_SERVICE, POWER_SERVICE, PRINT_SERVICE, RECEIVER_VISIBLE_TO_INSTANT_APPS, RESTRICTIONS_SERVICE, SEARCH_SERVICE, SENSOR_SERVICE, SHORTCUT_SERVICE, STORAGE_SERVICE, STORAGE_STATS_SERVICE, SYSTEM_HEALTH_SERVICE, TELECOM_SERVICE, TELEPHONY_SERVICE, TELEPHONY_SUBSCRIPTION_SERVICE, TEXT_CLASSIFICATION_SERVICE, TEXT_SERVICES_MANAGER_SERVICE, TV_INPUT_SERVICE, UI_MODE_SERVICE, USAGE_STATS_SERVICE, USB_SERVICE, USER_SERVICE, VIBRATOR_SERVICE, WALLPAPER_SERVICE, WIFI_AWARE_SERVICE, WIFI_P2P_SERVICE, WIFI_SERVICE, WINDOW_SERVICE

Fields inherited from interface android.content.ComponentCallbacks2

TRIM_MEMORY_BACKGROUND, TRIM_MEMORY_COMPLETE, TRIM_MEMORY_MODERATE, TRIM_MEMORY_RUNNING_CRITICAL, TRIM_MEMORY_RUNNING_LOW, TRIM_MEMORY_RUNNING_MODERATE, TRIM_MEMORY_UI_HIDDEN

Constructor Summary

Constructors

Constructor and Description
VisualizeCamera2Activity()

Method Summary

Modifier and Type	Method and Description
protected void	onCameraResolutionChange(int cameraWidth, int cameraHeight, int sensorOrientation) Called when the camera's resolution has changed.
protected void	onCreate(android.os.Bundle savedInstanceState)
protected void	onDrawFrame(android.view.SurfaceView view, android.graphics.Canvas canvas) Renders the visualizations.
protected void	processFrame(android.media.Image image) Process a single frame from the video feed.
protected abstract void	processImage(boofcv.struct.image.ImageBase image) Where all the image processing happens.
protected int	selectResolution(int widthTexture, int heightTexture, android.util.Size[] resolutions) Selects a resolution which has the number of pixels closest to the requested value
protected void	setImageType(boofcv.struct.image.ImageType type, boofcv.alg.color.ColorFormat colorFormat) Changes the type of image the camera frame is converted to
void	setThreadPoolSize(int threads) Specifies the number of threads in the thread-pool.
protected void	startCamera()
protected void	startCamera(android.view.ViewGroup layout, android.view.TextureView view)
protected void	startCameraTexture(android.view.TextureView view) After this function is called the camera will be start.
protected void	startCameraView(android.view.View view)
protected static void	videoToDisplayMatrix(int cameraWidth, int cameraHeight, int cameraRotation, int displayWidth, int displayHeight, int displayRotation, boolean stretchToFill, android.graphics.Matrix imageToView)

Methods inherited from class boofcv.android.camera2.SimpleCamera2Activity

cameraNominalFov, closeCamera, configureCamera, handleCameraOpenException, handleNoCameraSelected, handleOnImageAvailableException, onCameraDisconnected, onCameraOpened, onPause, onResume, openCamera, reopenCameraAtResolution, selectCamera, setVerbose

Methods inherited from class android.app.Activity

addContentView, attachBaseContext, closeContextMenu, closeOptionsMenu, createPendingResult, dismissDialog, dismissKeyboardShortcutsHelper, dispatchGenericMotionEvent, dispatchKeyEvent, dispatchKeyShortcutEvent, dispatchPopulateAccessibilityEvent, dispatchTouchEvent, dispatchTrackballEvent, dump, enterPictureInPictureMode, enterPictureInPictureMode, findViewById, finish, finishActivity, finishActivityFromChild, finishAffinity, finishAfterTransition, finishAndRemoveTask, finishFromChild, getActionBar, getApplication, getCallingActivity, getCallingPackage, getChangingConfigurations, getComponentName, getContentScene, getContentTransitionManager, getCurrentFocus, getFragmentManager, getIntent, getLastNonConfigurationInstance, getLayoutInflater, getLoaderManager, getLocalClassName, getMaxNumPictureInPictureActions, getMediaController, getMenuInflater, getParent, getParentActivityIntent, getPreferences, getReferrer, getRequestedOrientation, getSearchEvent, getSystemService, getTaskId, getTitle, getTitleColor, getVoiceInteractor, getVolumeControlStream, getWindow, getWindowManager, hasWindowFocus, invalidateOptionsMenu, isActivityTransitionRunning, isChangingConfigurations, isChild, isDestroyed, isFinishing, isImmersive, isInMultiWindowMode, isInPictureInPictureMode, isLocalVoiceInteractionSupported, isTaskRoot, isVoiceInteraction, isVoiceInteractionRoot, managedQuery, moveTaskToBack, navigateUpTo, navigateUpToFromChild, onActionModeFinished, onActionModeStarted, onActivityReenter, onActivityResult, onApplyThemeResource, onAttachedToWindow, onAttachFragment, onBackPressed, onChildTitleChanged, onConfigurationChanged, onContentChanged, onContextItemSelected, onContextMenuClosed, onCreate, onCreateContextMenu, onCreateDescription, onCreateDialog, onCreateDialog, onCreateNavigateUpTaskStack, onCreateOptionsMenu, onCreatePanelMenu, onCreatePanelView, onCreateThumbnail, onCreateView, onCreateView, onDestroy, onDetachedFromWindow, onEnterAnimationComplete, onGenericMotionEvent, onKeyDown, onKeyLongPress, onKeyMultiple, onKeyShortcut, onKeyUp, onLocalVoiceInteractionStarted, onLocalVoiceInteractionStopped, onLowMemory, onMenuItemSelected, onMenuOpened, onMultiWindowModeChanged, onMultiWindowModeChanged, onNavigateUp, onNavigateUpFromChild, onNewIntent, onOptionsItemSelected, onOptionsMenuClosed, onPanelClosed, onPictureInPictureModeChanged, onPictureInPictureModeChanged, onPostCreate, onPostCreate, onPostResume, onPrepareDialog, onPrepareDialog, onPrepareNavigateUpTaskStack, onPrepareOptionsMenu, onPreparePanel, onProvideAssistContent, onProvideAssistData, onProvideKeyboardShortcuts, onProvideReferrer, onRequestPermissionsResult, onRestart, onRestoreInstanceState, onRestoreInstanceState, onRetainNonConfigurationInstance, onSaveInstanceState, onSaveInstanceState, onSearchRequested, onSearchRequested, onStart, onStateNotSaved, onStop, onTitleChanged, onTouchEvent, onTrackballEvent, onTrimMemory, onUserInteraction, onUserLeaveHint, onVisibleBehindCanceled, onWindowAttributesChanged, onWindowFocusChanged, onWindowStartingActionMode, onWindowStartingActionMode, openContextMenu, openOptionsMenu, overridePendingTransition, postponeEnterTransition, recreate, registerForContextMenu, releaseInstance, removeDialog, reportFullyDrawn, requestDragAndDropPermissions, requestPermissions, requestShowKeyboardShortcuts, requestVisibleBehind, requestWindowFeature, runOnUiThread, setActionBar, setContentTransitionManager, setContentView, setContentView, setContentView, setDefaultKeyMode, setEnterSharedElementCallback, setExitSharedElementCallback, setFeatureDrawable, setFeatureDrawableAlpha, setFeatureDrawableResource, setFeatureDrawableUri, setFinishOnTouchOutside, setImmersive, setIntent, setMediaController, setPictureInPictureParams, setProgress, setProgressBarIndeterminate, setProgressBarIndeterminateVisibility, setProgressBarVisibility, setRequestedOrientation, setResult, setResult, setSecondaryProgress, setShowWhenLocked, setTaskDescription, setTheme, setTitle, setTitle, setTitleColor, setTurnScreenOn, setVisible, setVolumeControlStream, setVrModeEnabled, shouldShowRequestPermissionRationale, shouldUpRecreateTask, showAssist, showDialog, showDialog, showLockTaskEscapeMessage, startActionMode, startActionMode, startActivities, startActivities, startActivity, startActivity, startActivityForResult, startActivityForResult, startActivityFromChild, startActivityFromChild, startActivityFromFragment, startActivityFromFragment, startActivityIfNeeded, startActivityIfNeeded, startIntentSender, startIntentSender, startIntentSenderForResult, startIntentSenderForResult, startIntentSenderFromChild, startIntentSenderFromChild, startLocalVoiceInteraction, startLockTask, startManagingCursor, startNextMatchingActivity, startNextMatchingActivity, startPostponedEnterTransition, startSearch, stopLocalVoiceInteraction, stopLockTask, stopManagingCursor, takeKeyEvents, triggerSearch, unregisterForContextMenu

Methods inherited from class android.view.ContextThemeWrapper

applyOverrideConfiguration, getAssets, getResources, getTheme

Methods inherited from class android.content.ContextWrapper

bindService, checkCallingOrSelfPermission, checkCallingOrSelfUriPermission, checkCallingPermission, checkCallingUriPermission, checkPermission, checkSelfPermission, checkUriPermission, checkUriPermission, clearWallpaper, createConfigurationContext, createContextForSplit, createDeviceProtectedStorageContext, createDisplayContext, createPackageContext, databaseList, deleteDatabase, deleteFile, deleteSharedPreferences, enforceCallingOrSelfPermission, enforceCallingOrSelfUriPermission, enforceCallingPermission, enforceCallingUriPermission, enforcePermission, enforceUriPermission, enforceUriPermission, fileList, getApplicationContext, getApplicationInfo, getBaseContext, getCacheDir, getClassLoader, getCodeCacheDir, getContentResolver, getDatabasePath, getDataDir, getDir, getExternalCacheDir, getExternalCacheDirs, getExternalFilesDir, getExternalFilesDirs, getExternalMediaDirs, getFilesDir, getFileStreamPath, getMainLooper, getNoBackupFilesDir, getObbDir, getObbDirs, getPackageCodePath, getPackageManager, getPackageName, getPackageResourcePath, getSharedPreferences, getSystemServiceName, getWallpaper, getWallpaperDesiredMinimumHeight, getWallpaperDesiredMinimumWidth, grantUriPermission, isDeviceProtectedStorage, isRestricted, moveDatabaseFrom, moveSharedPreferencesFrom, openFileInput, openFileOutput, openOrCreateDatabase, openOrCreateDatabase, peekWallpaper, registerReceiver, registerReceiver, registerReceiver, registerReceiver, removeStickyBroadcast, removeStickyBroadcastAsUser, revokeUriPermission, revokeUriPermission, sendBroadcast, sendBroadcast, sendBroadcastAsUser, sendBroadcastAsUser, sendOrderedBroadcast, sendOrderedBroadcast, sendOrderedBroadcastAsUser, sendStickyBroadcast, sendStickyBroadcastAsUser, sendStickyOrderedBroadcast, sendStickyOrderedBroadcastAsUser, setWallpaper, setWallpaper, startForegroundService, startInstrumentation, startService, stopService, unbindService, unregisterReceiver

Methods inherited from class android.content.Context

getColor, getColorStateList, getDrawable, getString, getString, getSystemService, getText, obtainStyledAttributes, obtainStyledAttributes, obtainStyledAttributes, obtainStyledAttributes, registerComponentCallbacks, unregisterComponentCallbacks

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface android.view.Window.Callback

onPointerCaptureChanged

Field Detail

textureView

protected android.view.TextureView textureView

displayView

protected VisualizeCamera2Activity.DisplayView displayView

bitmapLock

protected final java.lang.Object bitmapLock

bitmap

protected android.graphics.Bitmap bitmap

bitmapTmp

protected byte[] bitmapTmp

threadQueue

protected java.util.concurrent.LinkedBlockingQueue threadQueue

threadPool

protected java.util.concurrent.ThreadPoolExecutor threadPool

imageToView

protected android.graphics.Matrix imageToView

Stores the transform from the video image to the view it is displayed on. Handles All those annoying rotations.

viewToImage

protected android.graphics.Matrix viewToImage

targetResolution

protected int targetResolution

showBitmap

protected boolean showBitmap

stretchToFill

protected boolean stretchToFill

visualizeOnlyMostRecent

protected boolean visualizeOnlyMostRecent

timeOfLastUpdated

protected volatile long timeOfLastUpdated

TIMING_WARM_UP

protected static final int TIMING_WARM_UP

See Also:

Constant Field Values

lockTiming

protected final java.lang.Object lockTiming

totalConverted

protected int totalConverted

periodConvert

protected final boofcv.misc.MovingAverage periodConvert

Constructor Detail

VisualizeCamera2Activity

public VisualizeCamera2Activity()

Method Detail

onCreate

protected void onCreate(android.os.Bundle savedInstanceState)

Overrides:

onCreate in class SimpleCamera2Activity

setThreadPoolSize

public void setThreadPoolSize(int threads)

Specifies the number of threads in the thread-pool. If this is set to a value greater than one you better know how to write concurrent code or else you're going to have a bad time.

startCamera

protected void startCamera(@NonNull
                           android.view.ViewGroup layout,
                           @Nullable
                           android.view.TextureView view)

Parameters:

layout - Where the visualization overlay will be placed inside of

view - If not null then this will be used to display the camera preview.

startCameraTexture

protected void startCameraTexture(@Nullable
                                  android.view.TextureView view)

Description copied from class: SimpleCamera2Activity

After this function is called the camera will be start. It might not start immediately and there can be a delay.

Overrides:

startCameraTexture in class SimpleCamera2Activity

Parameters:

view - The view the camera is displayed inside or null if not displayed

startCameraView

protected void startCameraView(@Nullable
                               android.view.View view)

Overrides:

startCameraView in class SimpleCamera2Activity

startCamera

protected void startCamera()

Overrides:

startCamera in class SimpleCamera2Activity

selectResolution

protected int selectResolution(int widthTexture,
                               int heightTexture,
                               android.util.Size[] resolutions)

Selects a resolution which has the number of pixels closest to the requested value

Overrides:

selectResolution in class SimpleCamera2Activity

Parameters:

widthTexture - Width of the texture the preview is displayed inside of. <= 0 if no view

heightTexture - Height of the texture the preview is displayed inside of. <= 0 if no view

resolutions - array of possible resolutions

Returns:

index of the resolution

onCameraResolutionChange

protected void onCameraResolutionChange(int cameraWidth,
                                        int cameraHeight,
                                        int sensorOrientation)

Description copied from class: SimpleCamera2Activity

Called when the camera's resolution has changed. This function can be called more than once each time a camera is opened, e.g. requested resolution does not match actual.

Overrides:

onCameraResolutionChange in class SimpleCamera2Activity

videoToDisplayMatrix

protected static void videoToDisplayMatrix(int cameraWidth,
                                           int cameraHeight,
                                           int cameraRotation,
                                           int displayWidth,
                                           int displayHeight,
                                           int displayRotation,
                                           boolean stretchToFill,
                                           android.graphics.Matrix imageToView)

setImageType

protected void setImageType(boofcv.struct.image.ImageType type,
                            boofcv.alg.color.ColorFormat colorFormat)

Changes the type of image the camera frame is converted to

processFrame

protected void processFrame(android.media.Image image)

Description copied from class: SimpleCamera2Activity

Process a single frame from the video feed. Image is automatically closed after this function exists. No need to invoke image.close() manually. All implementations of this function must run very fast. Less than 5 milliseconds is a good rule of thumb. If longer than that then you should spawn a thread and process the image inside of that.

Specified by:

processFrame in class SimpleCamera2Activity

processImage

protected abstract void processImage(boofcv.struct.image.ImageBase image)

Where all the image processing happens. If the number of threads is greater than one then this function can be called multiple times before previous calls have finished. WARNING: If the image type can change this must be before processing it.

Parameters:

image - The image which is to be processed. The image is owned by this function until it returns. After that the image and all it's data will be recycled. DO NOT SAVE A REFERENCE TO IT.

onDrawFrame

protected void onDrawFrame(android.view.SurfaceView view,
                           android.graphics.Canvas canvas)

Renders the visualizations. Override and invoke super to add your own